[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474615205622-30146-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:56:55 dist-test-slave-h6n0 krb5kdc[30159](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:56:55 dist-test-slave-h6n0 krb5kdc[30159](info): set up 2 sockets
Aug 11 18:56:55 dist-test-slave-h6n0 krb5kdc[30159](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260811 18:56:56.621567 30146 mini_kdc.cc:121] Time spent starting KDC: real 1.319s user 0.000s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474615205622-30146-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474615205622-30146-0/krb5kdc/test-user.keytab.
Aug 11 18:56:56 dist-test-slave-h6n0 krb5kdc[30159](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474616, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:56:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:56:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.112.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab.
Entry for principal kudu/127.29.112.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab.
I20260811 18:56:56.830456 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474615205622-30146-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474615205622-30146-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474615205622-30146-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.190:32829
--webserver_interface=127.29.112.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab
--principal=kudu/127.29.112.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:33177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.190:32829
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:56:57.313524 30183 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:56:57.313832 30183 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:56:57.314007 30183 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:56:57.321367 30183 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:56:57.321486 30183 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:56:57.321580 30183 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:56:57.321688 30183 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:56:57.321780 30183 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:56:57.331817 30183 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:33177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474615205622-30146-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474615205622-30146-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.112.190:32829
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab
--principal=kudu/127.29.112.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.190:32829
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.112.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474615205622-30146-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:56:57.333212 30183 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:56:57.334551 30183 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:56:57.352873 30190 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:56:57.353447 30189 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:56:57.353529 30193 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:56:57.354377 30183 server_base.cc:1034] running on GCE node
I20260811 18:56:57.355877 30183 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:56:57.357481 30183 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:56:57.358680 30183 hybrid_clock.cc:648] HybridClock initialized: now 1786474617358581 us; error 150 us; skew 500 ppm
I20260811 18:56:57.359122 30183 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:56:57 dist-test-slave-h6n0 krb5kdc[30159](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474617, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:56:57.367282 30183 init.cc:377] Logged in from keytab as kudu/127.29.112.190@KRBTEST.COM (short username kudu)
I20260811 18:56:57.369328 30183 webserver.cc:460] Webserver started at http://127.29.112.190:45525/ using document root <none> and password file <none>
I20260811 18:56:57.370111 30183 fs_manager.cc:359] Metadata directory not provided
I20260811 18:56:57.370278 30183 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:56:57.370702 30183 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:56:57.375659 30183 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474615205622-30146-0/minicluster-data/master-0/data/instance:
uuid: "f40ee0821828443fa4844548e7bb0eb3"
format_stamp: "Formatted at 2026-08-11 18:56:57 on dist-test-slave-h6n0"
server_key: "ed0d681afd7769e87bd36d6771030fa9"
server_key_iv: "009766c783094c130239fd2ab653bc6f"
server_key_version: "encryptionkey@0"
I20260811 18:56:57.376791 30183 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474615205622-30146-0/minicluster-data/master-0/wal/instance:
uuid: "f40ee0821828443fa4844548e7bb0eb3"
format_stamp: "Formatted at 2026-08-11 18:56:57 on dist-test-slave-h6n0"
server_key: "ed0d681afd7769e87bd36d6771030fa9"
server_key_iv: "009766c783094c130239fd2ab653bc6f"
server_key_version: "encryptionkey@0"
I20260811 18:56:57.383757 30183 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.006s
I20260811 18:56:57.387604 30206 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:56:57.388993 30183 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260811 18:56:57.389226 30183 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474615205622-30146-0/minicluster-data/master-0/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474615205622-30146-0/minicluster-data/master-0/wal
uuid: "f40ee0821828443fa4844548e7bb0eb3"
format_stamp: "Formatted at 2026-08-11 18:56:57 on dist-test-slave-h6n0"
server_key: "ed0d681afd7769e87bd36d6771030fa9"
server_key_iv: "009766c783094c130239fd2ab653bc6f"
server_key_version: "encryptionkey@0"
I20260811 18:56:57.389472 30183 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474615205622-30146-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474615205622-30146-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474615205622-30146-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:56:57.417599 30183 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:56:57.422488 30183 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:56:57.422839 30183 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:56:57.446821 30183 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.190:32829
I20260811 18:56:57.446856 30297 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.190:32829 every 8 connection(s)
I20260811 18:56:57.448262 30183 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
I20260811 18:56:57.452157 30298 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:56:57.456903 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 30183
I20260811 18:56:57.457273 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474615205622-30146-0/minicluster-data/master-0/wal/instance
I20260811 18:56:57.459280 30146 external_mini_cluster.cc:1428] Setting key c7274230d75d43c251f9474d5b292583
I20260811 18:56:57.465109 30298 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f40ee0821828443fa4844548e7bb0eb3: Bootstrap starting.
I20260811 18:56:57.468415 30298 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f40ee0821828443fa4844548e7bb0eb3: Neither blocks nor log segments found. Creating new log.
I20260811 18:56:57.469516 30298 log.cc:824] T 00000000000000000000000000000000 P f40ee0821828443fa4844548e7bb0eb3: Log is configured to *not* fsync() on all Append() calls
I20260811 18:56:57.472404 30298 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f40ee0821828443fa4844548e7bb0eb3: No bootstrap required, opened a new log
I20260811 18:56:57.481577 30298 raft_consensus.cc:348] T 00000000000000000000000000000000 P f40ee0821828443fa4844548e7bb0eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f40ee0821828443fa4844548e7bb0eb3" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 32829 } }
I20260811 18:56:57.482466 30298 raft_consensus.cc:374] T 00000000000000000000000000000000 P f40ee0821828443fa4844548e7bb0eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:56:57.482630 30298 raft_consensus.cc:729] T 00000000000000000000000000000000 P f40ee0821828443fa4844548e7bb0eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f40ee0821828443fa4844548e7bb0eb3, State: Initialized, Role: FOLLOWER
I20260811 18:56:57.483258 30298 consensus_queue.cc:260] T 00000000000000000000000000000000 P f40ee0821828443fa4844548e7bb0eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f40ee0821828443fa4844548e7bb0eb3" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 32829 } }
I20260811 18:56:57.483546 30298 raft_consensus.cc:388] T 00000000000000000000000000000000 P f40ee0821828443fa4844548e7bb0eb3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:56:57.483705 30298 raft_consensus.cc:482] T 00000000000000000000000000000000 P f40ee0821828443fa4844548e7bb0eb3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:56:57.483886 30298 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f40ee0821828443fa4844548e7bb0eb3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:56:57.486739 30298 raft_consensus.cc:504] T 00000000000000000000000000000000 P f40ee0821828443fa4844548e7bb0eb3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f40ee0821828443fa4844548e7bb0eb3" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 32829 } }
I20260811 18:56:57.487427 30298 leader_election.cc:302] T 00000000000000000000000000000000 P f40ee0821828443fa4844548e7bb0eb3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f40ee0821828443fa4844548e7bb0eb3; no voters:
I20260811 18:56:57.488183 30298 leader_election.cc:288] T 00000000000000000000000000000000 P f40ee0821828443fa4844548e7bb0eb3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:56:57.488315 30310 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f40ee0821828443fa4844548e7bb0eb3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:56:57.489814 30310 raft_consensus.cc:686] T 00000000000000000000000000000000 P f40ee0821828443fa4844548e7bb0eb3 [term 1 LEADER]: Becoming Leader. State: Replica: f40ee0821828443fa4844548e7bb0eb3, State: Running, Role: LEADER
I20260811 18:56:57.491047 30310 consensus_queue.cc:237] T 00000000000000000000000000000000 P f40ee0821828443fa4844548e7bb0eb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f40ee0821828443fa4844548e7bb0eb3" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 32829 } }
I20260811 18:56:57.491652 30298 sys_catalog.cc:564] T 00000000000000000000000000000000 P f40ee0821828443fa4844548e7bb0eb3 [sys.catalog]: configured and running, proceeding with master startup.
Aug 11 18:56:57 dist-test-slave-h6n0 krb5kdc[30159](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474616, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 18:56:57.495971 30311 sys_catalog.cc:455] T 00000000000000000000000000000000 P f40ee0821828443fa4844548e7bb0eb3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f40ee0821828443fa4844548e7bb0eb3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f40ee0821828443fa4844548e7bb0eb3" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 32829 } } }
I20260811 18:56:57.497138 30311 sys_catalog.cc:458] T 00000000000000000000000000000000 P f40ee0821828443fa4844548e7bb0eb3 [sys.catalog]: This master's current role is: LEADER
I20260811 18:56:57.497014 30312 sys_catalog.cc:455] T 00000000000000000000000000000000 P f40ee0821828443fa4844548e7bb0eb3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f40ee0821828443fa4844548e7bb0eb3. Latest consensus state: current_term: 1 leader_uuid: "f40ee0821828443fa4844548e7bb0eb3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f40ee0821828443fa4844548e7bb0eb3" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 32829 } } }
I20260811 18:56:57.498953 30312 sys_catalog.cc:458] T 00000000000000000000000000000000 P f40ee0821828443fa4844548e7bb0eb3 [sys.catalog]: This master's current role is: LEADER
I20260811 18:56:57.502992 30328 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:56:57.512537 30301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:56:57.463526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44186 (local address 127.29.112.190:32829)
0811 18:56:57.464522 (+ 996us) server_negotiation.cc:206] Beginning negotiation
0811 18:56:57.464541 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 18:56:57.465733 (+ 1192us) server_negotiation.cc:407] Connection header received
0811 18:56:57.477464 (+ 11731us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:56:57.477489 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:56:57.477945 (+ 456us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:56:57.478374 (+ 429us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:56:57.481420 (+ 3046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:56:57.483251 (+ 1831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:56:57.486023 (+ 2772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:56:57.486829 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:56:57.487048 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:56:57.494086 (+ 7038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:56:57.494099 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:56:57.494120 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:56:57.494166 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:56:57.505195 (+ 11029us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:56:57.506767 (+ 1572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:56:57.506782 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:56:57.506794 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:56:57.507126 (+ 332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:56:57.507743 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:56:57.507751 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:56:57.507757 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:56:57.508956 (+ 1199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:56:57.509378 (+ 422us) server_negotiation.cc:1035] Waiting for connection context
0811 18:56:57.510431 (+ 1053us) server_negotiation.cc:299] Negotiation successful
0811 18:56:57.510954 (+ 523us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":709,"thread_start_us":472,"threads_started":1}
I20260811 18:56:57.513368 30328 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:56:57.527175 30328 catalog_manager.cc:1347] Generated new cluster ID: 077de5b3967e420186987178801d5df2
I20260811 18:56:57.527326 30328 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:56:57.544564 30328 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:56:57.546947 30328 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:56:57.557803 30328 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f40ee0821828443fa4844548e7bb0eb3: Generated new TSK 0
I20260811 18:56:57.559587 30328 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.112.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.129.keytab.
Entry for principal kudu/127.29.112.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.129.keytab.
I20260811 18:56:57.623579 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474615205622-30146-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474615205622-30146-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474615205622-30146-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474615205622-30146-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.129:0
--local_ip_for_outbound_sockets=127.29.112.129
--webserver_interface=127.29.112.129
--webserver_port=0
--tserver_master_addrs=127.29.112.190:32829
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.129.keytab
--principal=kudu/127.29.112.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:33177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:56:58.034464 30348 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:56:58.034857 30348 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:56:58.035049 30348 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:56:58.042598 30348 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:56:58.042724 30348 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:56:58.042918 30348 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.129
I20260811 18:56:58.054049 30348 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:33177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474615205622-30146-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474615205622-30146-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.129.keytab
--principal=kudu/127.29.112.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474615205622-30146-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.112.129
--webserver_port=0
--tserver_master_addrs=127.29.112.190:32829
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.129
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474615205622-30146-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:56:58.055411 30348 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:56:58.056807 30348 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:56:58.072463 30356 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:56:58.072261 30359 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:56:58.072374 30363 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:56:58.076138 30348 server_base.cc:1034] running on GCE node
I20260811 18:56:58.076885 30348 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:56:58.077958 30348 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:56:58.079200 30348 hybrid_clock.cc:648] HybridClock initialized: now 1786474618079143 us; error 49 us; skew 500 ppm
I20260811 18:56:58.079571 30348 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:56:58 dist-test-slave-h6n0 krb5kdc[30159](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474618, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:56:58.083930 30348 init.cc:377] Logged in from keytab as kudu/127.29.112.129@KRBTEST.COM (short username kudu)
I20260811 18:56:58.085618 30348 webserver.cc:460] Webserver started at http://127.29.112.129:39409/ using document root <none> and password file <none>
I20260811 18:56:58.086330 30348 fs_manager.cc:359] Metadata directory not provided
I20260811 18:56:58.086438 30348 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:56:58.086803 30348 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:56:58.089855 30348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474615205622-30146-0/minicluster-data/ts-0/data/instance:
uuid: "033f9d8941e7478b8985a404580c86c6"
format_stamp: "Formatted at 2026-08-11 18:56:58 on dist-test-slave-h6n0"
server_key: "1dfdd65c6bf6bf229e2698a3b3256e5d"
server_key_iv: "51dda6dea77bbd16aac33a06d3178838"
server_key_version: "encryptionkey@0"
I20260811 18:56:58.090848 30348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474615205622-30146-0/minicluster-data/ts-0/wal/instance:
uuid: "033f9d8941e7478b8985a404580c86c6"
format_stamp: "Formatted at 2026-08-11 18:56:58 on dist-test-slave-h6n0"
server_key: "1dfdd65c6bf6bf229e2698a3b3256e5d"
server_key_iv: "51dda6dea77bbd16aac33a06d3178838"
server_key_version: "encryptionkey@0"
I20260811 18:56:58.096859 30348 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.003s
I20260811 18:56:58.101300 30371 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:56:58.103334 30348 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260811 18:56:58.103544 30348 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474615205622-30146-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474615205622-30146-0/minicluster-data/ts-0/wal
uuid: "033f9d8941e7478b8985a404580c86c6"
format_stamp: "Formatted at 2026-08-11 18:56:58 on dist-test-slave-h6n0"
server_key: "1dfdd65c6bf6bf229e2698a3b3256e5d"
server_key_iv: "51dda6dea77bbd16aac33a06d3178838"
server_key_version: "encryptionkey@0"
I20260811 18:56:58.103773 30348 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474615205622-30146-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474615205622-30146-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474615205622-30146-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:56:58.140316 30348 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:56:58.146648 30348 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:56:58.146929 30348 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:56:58.147924 30348 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:56:58.149611 30348 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:56:58.149690 30348 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:56:58.149787 30348 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:56:58.149833 30348 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:56:58.187855 30348 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.129:41859
I20260811 18:56:58.187965 30578 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.129:41859 every 8 connection(s)
I20260811 18:56:58.189234 30348 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474615205622-30146-0/minicluster-data/ts-0/data/info.pb
I20260811 18:56:58.193553 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 30348
I20260811 18:56:58.193799 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474615205622-30146-0/minicluster-data/ts-0/wal/instance
I20260811 18:56:58.194420 30146 external_mini_cluster.cc:1428] Setting key 37d7fc7641dc9508b40cb289990f4477
Aug 11 18:56:58 dist-test-slave-h6n0 krb5kdc[30159](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474618, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.129@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 18:56:58.215791 30581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:56:58.192101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.129:36385 (local address 127.29.112.190:32829)
0811 18:56:58.192650 (+ 549us) server_negotiation.cc:206] Beginning negotiation
0811 18:56:58.192656 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:56:58.193754 (+ 1098us) server_negotiation.cc:407] Connection header received
0811 18:56:58.196024 (+ 2270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:56:58.196030 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:56:58.196106 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:56:58.196215 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:56:58.198280 (+ 2065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:56:58.199891 (+ 1611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:56:58.202796 (+ 2905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:56:58.203857 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:56:58.204044 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:56:58.209384 (+ 5340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:56:58.209390 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:56:58.209393 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:56:58.209429 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:56:58.212483 (+ 3054us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:56:58.213284 (+ 801us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:56:58.213289 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:56:58.213293 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:56:58.213438 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:56:58.213896 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:56:58.213905 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:56:58.213907 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:56:58.214197 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:56:58.214357 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 18:56:58.215188 (+ 831us) server_negotiation.cc:299] Negotiation successful
0811 18:56:58.215426 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":417,"thread_start_us":255,"threads_started":1}
I20260811 18:56:58.216818 30583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:56:58.192492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:32829 (local address 127.29.112.129:36385)
0811 18:56:58.193433 (+ 941us) negotiation.cc:107] Waiting for socket to connect
0811 18:56:58.193474 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 18:56:58.195737 (+ 2263us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:56:58.196529 (+ 792us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:56:58.196537 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:56:58.197085 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:56:58.197643 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:56:58.197887 (+ 244us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:56:58.200165 (+ 2278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:56:58.200174 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:56:58.202460 (+ 2286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:56:58.202484 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:56:58.204167 (+ 1683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:56:58.204173 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:56:58.204441 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:56:58.205511 (+ 1070us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:56:58.205548 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:56:58.209044 (+ 3496us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:56:58.212646 (+ 3602us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:56:58.212657 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:56:58.212674 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:56:58.213148 (+ 474us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:56:58.213558 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:56:58.213563 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:56:58.213567 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:56:58.213770 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:56:58.214332 (+ 562us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:56:58.214340 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:56:58.214795 (+ 455us) client_negotiation.cc:769] Sending connection context
0811 18:56:58.215073 (+ 278us) client_negotiation.cc:240] Negotiation successful
0811 18:56:58.215636 (+ 563us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":667,"thread_start_us":382,"threads_started":1}
I20260811 18:56:58.218803 30579 heartbeater.cc:347] Connected to a master server at 127.29.112.190:32829
I20260811 18:56:58.219149 30579 heartbeater.cc:464] Registering TS with master...
I20260811 18:56:58.220168 30579 heartbeater.cc:511] Master 127.29.112.190:32829 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.29.112.130@KRBTEST.COM; defaulting to no policy
I20260811 18:56:58.223253 30234 ts_manager.cc:194] Registered new tserver with Master: 033f9d8941e7478b8985a404580c86c6 (127.29.112.129:41859)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.130@KRBTEST.COM" created.
I20260811 18:56:58.225762 30234 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.129@KRBTEST.COM'} at 127.29.112.129:36385
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.130.keytab.
Entry for principal kudu/127.29.112.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.130.keytab.
I20260811 18:56:58.260893 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474615205622-30146-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474615205622-30146-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474615205622-30146-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474615205622-30146-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.130:0
--local_ip_for_outbound_sockets=127.29.112.130
--webserver_interface=127.29.112.130
--webserver_port=0
--tserver_master_addrs=127.29.112.190:32829
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.130.keytab
--principal=kudu/127.29.112.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:33177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:56:58.649439 30588 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:56:58.649760 30588 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:56:58.649931 30588 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:56:58.657805 30588 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:56:58.657940 30588 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:56:58.658134 30588 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.130
I20260811 18:56:58.669133 30588 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:33177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474615205622-30146-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474615205622-30146-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.130.keytab
--principal=kudu/127.29.112.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474615205622-30146-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.112.130
--webserver_port=0
--tserver_master_addrs=127.29.112.190:32829
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.130
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474615205622-30146-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:56:58.670634 30588 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:56:58.672122 30588 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:56:58.688349 30600 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:56:58.687992 30594 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:56:58.688376 30596 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:56:58.689256 30588 server_base.cc:1034] running on GCE node
I20260811 18:56:58.689915 30588 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:56:58.691048 30588 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:56:58.692320 30588 hybrid_clock.cc:648] HybridClock initialized: now 1786474618692226 us; error 98 us; skew 500 ppm
I20260811 18:56:58.692678 30588 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:56:58 dist-test-slave-h6n0 krb5kdc[30159](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474618, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:56:58.696481 30588 init.cc:377] Logged in from keytab as kudu/127.29.112.130@KRBTEST.COM (short username kudu)
I20260811 18:56:58.698076 30588 webserver.cc:460] Webserver started at http://127.29.112.130:38897/ using document root <none> and password file <none>
I20260811 18:56:58.698892 30588 fs_manager.cc:359] Metadata directory not provided
I20260811 18:56:58.699064 30588 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:56:58.699365 30588 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:56:58.702916 30588 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474615205622-30146-0/minicluster-data/ts-1/data/instance:
uuid: "09e8db0ef0d745e3b0bffd65861965d5"
format_stamp: "Formatted at 2026-08-11 18:56:58 on dist-test-slave-h6n0"
server_key: "f8fbe69d1aba11ef2281c617b45a1ae4"
server_key_iv: "3a475588f2a5938ab8e93f2afadc7f7a"
server_key_version: "encryptionkey@0"
I20260811 18:56:58.704133 30588 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474615205622-30146-0/minicluster-data/ts-1/wal/instance:
uuid: "09e8db0ef0d745e3b0bffd65861965d5"
format_stamp: "Formatted at 2026-08-11 18:56:58 on dist-test-slave-h6n0"
server_key: "f8fbe69d1aba11ef2281c617b45a1ae4"
server_key_iv: "3a475588f2a5938ab8e93f2afadc7f7a"
server_key_version: "encryptionkey@0"
I20260811 18:56:58.711040 30588 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260811 18:56:58.717057 30615 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:56:58.718825 30588 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.006s sys 0.000s
I20260811 18:56:58.718994 30588 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474615205622-30146-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474615205622-30146-0/minicluster-data/ts-1/wal
uuid: "09e8db0ef0d745e3b0bffd65861965d5"
format_stamp: "Formatted at 2026-08-11 18:56:58 on dist-test-slave-h6n0"
server_key: "f8fbe69d1aba11ef2281c617b45a1ae4"
server_key_iv: "3a475588f2a5938ab8e93f2afadc7f7a"
server_key_version: "encryptionkey@0"
I20260811 18:56:58.719190 30588 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474615205622-30146-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474615205622-30146-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474615205622-30146-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:56:58.732497 30588 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:56:58.737078 30588 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:56:58.737332 30588 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:56:58.738235 30588 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:56:58.739667 30588 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:56:58.739748 30588 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:56:58.739845 30588 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:56:58.739898 30588 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:56:58.783500 30588 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.130:34113
I20260811 18:56:58.783612 30811 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.130:34113 every 8 connection(s)
I20260811 18:56:58.784816 30588 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474615205622-30146-0/minicluster-data/ts-1/data/info.pb
I20260811 18:56:58.787631 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 30588
I20260811 18:56:58.787855 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474615205622-30146-0/minicluster-data/ts-1/wal/instance
I20260811 18:56:58.788415 30146 external_mini_cluster.cc:1428] Setting key d2d1ccb730903bc508abec3d9e7030ce
Aug 11 18:56:58 dist-test-slave-h6n0 krb5kdc[30159](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474618, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.130@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 18:56:58.809199 30822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:56:58.787704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.130:34629 (local address 127.29.112.190:32829)
0811 18:56:58.788252 (+ 548us) server_negotiation.cc:206] Beginning negotiation
0811 18:56:58.788257 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:56:58.789446 (+ 1189us) server_negotiation.cc:407] Connection header received
0811 18:56:58.791885 (+ 2439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:56:58.791891 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:56:58.791982 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:56:58.792098 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:56:58.793903 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:56:58.794790 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:56:58.796412 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:56:58.796840 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:56:58.796980 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:56:58.801535 (+ 4555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:56:58.801543 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:56:58.801547 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:56:58.801606 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:56:58.805595 (+ 3989us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:56:58.806471 (+ 876us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:56:58.806477 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:56:58.806483 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:56:58.806689 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:56:58.807290 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:56:58.807298 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:56:58.807302 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:56:58.807634 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:56:58.807810 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0811 18:56:58.808571 (+ 761us) server_negotiation.cc:299] Negotiation successful
0811 18:56:58.808817 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":278,"threads_started":1}
WARNING: no policy specified for kudu/127.29.112.131@KRBTEST.COM; defaulting to no policy
I20260811 18:56:58.810436 30823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:56:58.788362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:32829 (local address 127.29.112.130:34629)
0811 18:56:58.789228 (+ 866us) negotiation.cc:107] Waiting for socket to connect
0811 18:56:58.789293 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0811 18:56:58.791614 (+ 2321us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:56:58.792380 (+ 766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:56:58.792388 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:56:58.792995 (+ 607us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:56:58.793700 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:56:58.793717 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:56:58.794977 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:56:58.794982 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:56:58.796237 (+ 1255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:56:58.796254 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:56:58.797077 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:56:58.797084 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:56:58.797316 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:56:58.798124 (+ 808us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:56:58.798149 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:56:58.801072 (+ 2923us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:56:58.805798 (+ 4726us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:56:58.805809 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:56:58.805833 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:56:58.806323 (+ 490us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:56:58.806901 (+ 578us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:56:58.806914 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:56:58.806919 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:56:58.807151 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:56:58.807791 (+ 640us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:56:58.807800 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:56:58.808192 (+ 392us) client_negotiation.cc:769] Sending connection context
0811 18:56:58.808485 (+ 293us) client_negotiation.cc:240] Negotiation successful
0811 18:56:58.809091 (+ 606us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":580,"thread_start_us":355,"threads_started":1}
I20260811 18:56:58.812436 30813 heartbeater.cc:347] Connected to a master server at 127.29.112.190:32829
I20260811 18:56:58.812789 30813 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.131@KRBTEST.COM" created.
I20260811 18:56:58.813683 30813 heartbeater.cc:511] Master 127.29.112.190:32829 requested a full tablet report, sending...
I20260811 18:56:58.815505 30234 ts_manager.cc:194] Registered new tserver with Master: 09e8db0ef0d745e3b0bffd65861965d5 (127.29.112.130:34113)
I20260811 18:56:58.816558 30234 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.130@KRBTEST.COM'} at 127.29.112.130:34629
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.131.keytab.
Entry for principal kudu/127.29.112.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.131.keytab.
I20260811 18:56:58.842788 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474615205622-30146-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474615205622-30146-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474615205622-30146-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474615205622-30146-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.131:0
--local_ip_for_outbound_sockets=127.29.112.131
--webserver_interface=127.29.112.131
--webserver_port=0
--tserver_master_addrs=127.29.112.190:32829
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.131.keytab
--principal=kudu/127.29.112.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:33177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 18:56:59.229763 30579 heartbeater.cc:503] Master 127.29.112.190:32829 was elected leader, sending a full tablet report...
W20260811 18:56:59.272786 30828 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:56:59.273150 30828 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:56:59.273365 30828 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:56:59.285143 30828 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:56:59.285306 30828 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:56:59.285555 30828 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.131
I20260811 18:56:59.305140 30828 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:33177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474615205622-30146-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474615205622-30146-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.131.keytab
--principal=kudu/127.29.112.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474615205622-30146-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.112.131
--webserver_port=0
--tserver_master_addrs=127.29.112.190:32829
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.131
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474615205622-30146-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:56:59.308573 30828 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:56:59.310387 30828 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:56:59.325063 30837 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:56:59.325107 30835 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:56:59.325572 30833 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:56:59.326061 30828 server_base.cc:1034] running on GCE node
I20260811 18:56:59.326809 30828 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:56:59.327777 30828 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:56:59.329043 30828 hybrid_clock.cc:648] HybridClock initialized: now 1786474619329007 us; error 53 us; skew 500 ppm
I20260811 18:56:59.329408 30828 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:56:59 dist-test-slave-h6n0 krb5kdc[30159](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474619, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:56:59.333331 30828 init.cc:377] Logged in from keytab as kudu/127.29.112.131@KRBTEST.COM (short username kudu)
I20260811 18:56:59.335240 30828 webserver.cc:460] Webserver started at http://127.29.112.131:36359/ using document root <none> and password file <none>
I20260811 18:56:59.335966 30828 fs_manager.cc:359] Metadata directory not provided
I20260811 18:56:59.336124 30828 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:56:59.336455 30828 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:56:59.339905 30828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474615205622-30146-0/minicluster-data/ts-2/data/instance:
uuid: "d987ac1b12014561a19dc900bf499463"
format_stamp: "Formatted at 2026-08-11 18:56:59 on dist-test-slave-h6n0"
server_key: "2ac3b3a9a60f510d8e16a20ed1775ae6"
server_key_iv: "46eb1a23e979dcf0516dcb968cb63998"
server_key_version: "encryptionkey@0"
I20260811 18:56:59.340713 30828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474615205622-30146-0/minicluster-data/ts-2/wal/instance:
uuid: "d987ac1b12014561a19dc900bf499463"
format_stamp: "Formatted at 2026-08-11 18:56:59 on dist-test-slave-h6n0"
server_key: "2ac3b3a9a60f510d8e16a20ed1775ae6"
server_key_iv: "46eb1a23e979dcf0516dcb968cb63998"
server_key_version: "encryptionkey@0"
I20260811 18:56:59.346689 30828 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.006s
I20260811 18:56:59.350777 30850 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:56:59.352455 30828 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:56:59.352764 30828 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474615205622-30146-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474615205622-30146-0/minicluster-data/ts-2/wal
uuid: "d987ac1b12014561a19dc900bf499463"
format_stamp: "Formatted at 2026-08-11 18:56:59 on dist-test-slave-h6n0"
server_key: "2ac3b3a9a60f510d8e16a20ed1775ae6"
server_key_iv: "46eb1a23e979dcf0516dcb968cb63998"
server_key_version: "encryptionkey@0"
I20260811 18:56:59.353000 30828 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474615205622-30146-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474615205622-30146-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474615205622-30146-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:56:59.369252 30828 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:56:59.373184 30828 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:56:59.373476 30828 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:56:59.374403 30828 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:56:59.375900 30828 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:56:59.376008 30828 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:56:59.376170 30828 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:56:59.376248 30828 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:56:59.417145 30828 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.131:39385
I20260811 18:56:59.417179 31051 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.131:39385 every 8 connection(s)
I20260811 18:56:59.418637 30828 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474615205622-30146-0/minicluster-data/ts-2/data/info.pb
I20260811 18:56:59.418927 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 30828
I20260811 18:56:59.419130 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474615205622-30146-0/minicluster-data/ts-2/wal/instance
I20260811 18:56:59.419612 30146 external_mini_cluster.cc:1428] Setting key 00e999838c257b27a43c8824fb5d70cc
Aug 11 18:56:59 dist-test-slave-h6n0 krb5kdc[30159](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474619, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.131@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 18:56:59.443046 31056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:56:59.423541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.131:42187 (local address 127.29.112.190:32829)
0811 18:56:59.425655 (+ 2114us) server_negotiation.cc:206] Beginning negotiation
0811 18:56:59.425660 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:56:59.425678 (+ 18us) server_negotiation.cc:407] Connection header received
0811 18:56:59.426781 (+ 1103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:56:59.426786 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:56:59.426884 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:56:59.427164 (+ 280us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:56:59.429178 (+ 2014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:56:59.430168 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:56:59.431838 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:56:59.432293 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:56:59.432420 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:56:59.436758 (+ 4338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:56:59.436764 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:56:59.436768 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:56:59.436808 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:56:59.439580 (+ 2772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:56:59.440339 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:56:59.440344 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:56:59.440346 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:56:59.440491 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:56:59.441019 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:56:59.441026 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:56:59.441029 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:56:59.441436 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:56:59.441564 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 18:56:59.442311 (+ 747us) server_negotiation.cc:299] Negotiation successful
0811 18:56:59.442479 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2014,"thread_start_us":1892,"threads_started":1}
I20260811 18:56:59.443713 31060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:56:59.423553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:32829 (local address 127.29.112.131:42187)
0811 18:56:59.424478 (+ 925us) negotiation.cc:107] Waiting for socket to connect
0811 18:56:59.424521 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 18:56:59.426245 (+ 1724us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:56:59.427457 (+ 1212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:56:59.427468 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:56:59.428228 (+ 760us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:56:59.428939 (+ 711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:56:59.428960 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:56:59.430355 (+ 1395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:56:59.430359 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:56:59.431672 (+ 1313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:56:59.431684 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:56:59.432513 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:56:59.432519 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:56:59.432719 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:56:59.433631 (+ 912us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:56:59.433653 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:56:59.436485 (+ 2832us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:56:59.439723 (+ 3238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:56:59.439730 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:56:59.439742 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:56:59.440194 (+ 452us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:56:59.440600 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:56:59.440603 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:56:59.440605 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:56:59.440793 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:56:59.441604 (+ 811us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:56:59.441609 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:56:59.441923 (+ 314us) client_negotiation.cc:769] Sending connection context
0811 18:56:59.442171 (+ 248us) client_negotiation.cc:240] Negotiation successful
0811 18:56:59.442623 (+ 452us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":578,"thread_start_us":592,"threads_started":1}
I20260811 18:56:59.445106 31053 heartbeater.cc:347] Connected to a master server at 127.29.112.190:32829
I20260811 18:56:59.445399 31053 heartbeater.cc:464] Registering TS with master...
I20260811 18:56:59.446139 31053 heartbeater.cc:511] Master 127.29.112.190:32829 requested a full tablet report, sending...
I20260811 18:56:59.447883 30234 ts_manager.cc:194] Registered new tserver with Master: d987ac1b12014561a19dc900bf499463 (127.29.112.131:39385)
I20260811 18:56:59.449018 30234 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.131@KRBTEST.COM'} at 127.29.112.131:42187
I20260811 18:56:59.449276 30146 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 18:56:59 dist-test-slave-h6n0 krb5kdc[30159](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474619, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 18:56:59 dist-test-slave-h6n0 krb5kdc[30159](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474619, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.29.112.129@KRBTEST.COM
I20260811 18:56:59.501278 31070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:56:59.484155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37402 (local address 127.29.112.129:41859)
0811 18:56:59.485245 (+ 1090us) server_negotiation.cc:206] Beginning negotiation
0811 18:56:59.485252 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:56:59.485279 (+ 27us) server_negotiation.cc:407] Connection header received
0811 18:56:59.485342 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:56:59.485348 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:56:59.485674 (+ 326us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:56:59.485982 (+ 308us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:56:59.487481 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:56:59.488540 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:56:59.489848 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:56:59.490387 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:56:59.490489 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:56:59.495300 (+ 4811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:56:59.495312 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:56:59.495326 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:56:59.495359 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:56:59.498172 (+ 2813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:56:59.498956 (+ 784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:56:59.498972 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:56:59.498974 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:56:59.499173 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:56:59.499626 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:56:59.499630 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:56:59.499632 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:56:59.500012 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:56:59.500263 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0811 18:56:59.500650 (+ 387us) server_negotiation.cc:299] Negotiation successful
0811 18:56:59.500853 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":949,"thread_start_us":809,"threads_started":1}
W20260811 18:56:59.502737 30424 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37402
Aug 11 18:56:59 dist-test-slave-h6n0 krb5kdc[30159](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474619, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 18:56:59 dist-test-slave-h6n0 krb5kdc[30159](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474619, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.129@KRBTEST.COM
I20260811 18:56:59.561508 31070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:56:59.543119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37410 (local address 127.29.112.129:41859)
0811 18:56:59.543328 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 18:56:59.543333 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:56:59.543626 (+ 293us) server_negotiation.cc:407] Connection header received
0811 18:56:59.543824 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:56:59.543828 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:56:59.543929 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:56:59.544014 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:56:59.546110 (+ 2096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:56:59.547113 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:56:59.549377 (+ 2264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:56:59.549832 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:56:59.549936 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:56:59.555596 (+ 5660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:56:59.555601 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:56:59.555605 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:56:59.555637 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:56:59.558686 (+ 3049us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:56:59.559598 (+ 912us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:56:59.559602 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:56:59.559604 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:56:59.559744 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:56:59.560179 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:56:59.560182 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:56:59.560184 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:56:59.560422 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:56:59.560607 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0811 18:56:59.561048 (+ 441us) server_negotiation.cc:299] Negotiation successful
0811 18:56:59.561216 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260811 18:56:59.563409 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 30348
I20260811 18:56:59.582557 31070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:56:59.564482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37412 (local address 127.29.112.129:41859)
0811 18:56:59.564701 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0811 18:56:59.564710 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:56:59.567037 (+ 2327us) server_negotiation.cc:407] Connection header received
0811 18:56:59.567178 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:56:59.567186 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:56:59.567293 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:56:59.567435 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:56:59.568639 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:56:59.570135 (+ 1496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:56:59.571502 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:56:59.572436 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:56:59.572597 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:56:59.574816 (+ 2219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:56:59.574826 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:56:59.574834 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:56:59.574882 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:56:59.579429 (+ 4547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:56:59.580092 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:56:59.580101 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:56:59.580108 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:56:59.580321 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:56:59.580692 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:56:59.580700 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:56:59.580707 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:56:59.581137 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:56:59.581450 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0811 18:56:59.581614 (+ 164us) server_negotiation.cc:299] Negotiation successful
0811 18:56:59.581887 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 18:56:59.583521 30571 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:56:59.671893 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 30348
I20260811 18:56:59.703478 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 30588
Aug 11 18:56:59 dist-test-slave-h6n0 krb5kdc[30159](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474619, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.130@KRBTEST.COM
I20260811 18:56:59.722476 31082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:56:59.704548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44184 (local address 127.29.112.130:34113)
0811 18:56:59.705056 (+ 508us) server_negotiation.cc:206] Beginning negotiation
0811 18:56:59.705063 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:56:59.705088 (+ 25us) server_negotiation.cc:407] Connection header received
0811 18:56:59.705147 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:56:59.705153 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:56:59.705360 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:56:59.705530 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:56:59.706816 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:56:59.707820 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:56:59.709080 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:56:59.709609 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:56:59.709758 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:56:59.714703 (+ 4945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:56:59.714715 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:56:59.714729 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:56:59.714762 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:56:59.719019 (+ 4257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:56:59.719705 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:56:59.719718 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:56:59.719723 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:56:59.719883 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:56:59.720694 (+ 811us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:56:59.720698 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:56:59.720700 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:56:59.721076 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:56:59.721342 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0811 18:56:59.721766 (+ 424us) server_negotiation.cc:299] Negotiation successful
0811 18:56:59.722021 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":246,"threads_started":1}
I20260811 18:56:59.723896 30806 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:56:59.808844 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 30588
I20260811 18:56:59.837299 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 30828
Aug 11 18:56:59 dist-test-slave-h6n0 krb5kdc[30159](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474619, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.131@KRBTEST.COM
I20260811 18:56:59.854947 31086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:56:59.838405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35276 (local address 127.29.112.131:39385)
0811 18:56:59.839036 (+ 631us) server_negotiation.cc:206] Beginning negotiation
0811 18:56:59.839043 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:56:59.839072 (+ 29us) server_negotiation.cc:407] Connection header received
0811 18:56:59.839136 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:56:59.839142 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:56:59.839338 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:56:59.839529 (+ 191us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:56:59.840946 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:56:59.841970 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:56:59.843397 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:56:59.843957 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:56:59.844108 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:56:59.849232 (+ 5124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:56:59.849243 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:56:59.849257 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:56:59.849291 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:56:59.852132 (+ 2841us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:56:59.852788 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:56:59.852795 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:56:59.852797 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:56:59.852930 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:56:59.853318 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:56:59.853323 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:56:59.853326 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:56:59.853671 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:56:59.853982 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0811 18:56:59.854301 (+ 319us) server_negotiation.cc:299] Negotiation successful
0811 18:56:59.854592 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":496,"thread_start_us":343,"threads_started":1}
I20260811 18:56:59.856359 31044 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:56:59.940706 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 30828
I20260811 18:56:59.963850 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 30183
I20260811 18:56:59.965476 30289 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:57:00.070082 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 30183
2026-08-11T18:57:00Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (4829 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474615205622-30146-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:57:00 dist-test-slave-h6n0 krb5kdc[31094](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:57:00 dist-test-slave-h6n0 krb5kdc[31094](info): set up 2 sockets
Aug 11 18:57:00 dist-test-slave-h6n0 krb5kdc[31094](info): commencing operation
krb5kdc: starting...
W20260811 18:57:01.435950 30146 mini_kdc.cc:121] Time spent starting KDC: real 1.314s user 0.001s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474615205622-30146-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474615205622-30146-0/krb5kdc/test-user.keytab.
Aug 11 18:57:01 dist-test-slave-h6n0 krb5kdc[31094](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474621, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:57:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:57:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.112.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab.
Entry for principal kudu/127.29.112.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab.
I20260811 18:57:01.659375 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474615205622-30146-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474615205622-30146-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474615205622-30146-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.190:38899
--webserver_interface=127.29.112.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab
--principal=kudu/127.29.112.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:33337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.190:38899
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:57:01.995302 31116 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:57:01.995690 31116 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:57:01.995862 31116 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:57:02.003551 31116 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:57:02.003720 31116 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:57:02.003835 31116 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:57:02.003947 31116 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:57:02.004036 31116 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:57:02.014492 31116 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:33337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474615205622-30146-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474615205622-30146-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.112.190:38899
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab
--principal=kudu/127.29.112.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.190:38899
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.112.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474615205622-30146-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:57:02.015920 31116 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:57:02.017663 31116 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:57:02.030139 31129 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:02.030102 31130 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:02.030179 31132 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:57:02.031646 31116 server_base.cc:1034] running on GCE node
I20260811 18:57:02.032799 31116 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:57:02.034276 31116 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:57:02.035527 31116 hybrid_clock.cc:648] HybridClock initialized: now 1786474622035522 us; error 106 us; skew 500 ppm
I20260811 18:57:02.035892 31116 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:57:02 dist-test-slave-h6n0 krb5kdc[31094](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474622, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:57:02.039268 31116 init.cc:377] Logged in from keytab as kudu/127.29.112.190@KRBTEST.COM (short username kudu)
I20260811 18:57:02.040874 31116 webserver.cc:460] Webserver started at http://127.29.112.190:35605/ using document root <none> and password file <none>
I20260811 18:57:02.041566 31116 fs_manager.cc:359] Metadata directory not provided
I20260811 18:57:02.041715 31116 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:57:02.042037 31116 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:57:02.045071 31116 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474615205622-30146-0/minicluster-data/master-0/data/instance:
uuid: "209fb7d1be7f44a8aecf2cd0edaeef28"
format_stamp: "Formatted at 2026-08-11 18:57:02 on dist-test-slave-h6n0"
server_key: "8d99c68be4e0365a40d4df6c7fbb8f6d"
server_key_iv: "82da1d2abd4c0c3c56adfb88686ef27e"
server_key_version: "encryptionkey@0"
I20260811 18:57:02.045807 31116 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474615205622-30146-0/minicluster-data/master-0/wal/instance:
uuid: "209fb7d1be7f44a8aecf2cd0edaeef28"
format_stamp: "Formatted at 2026-08-11 18:57:02 on dist-test-slave-h6n0"
server_key: "8d99c68be4e0365a40d4df6c7fbb8f6d"
server_key_iv: "82da1d2abd4c0c3c56adfb88686ef27e"
server_key_version: "encryptionkey@0"
I20260811 18:57:02.051036 31116 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 18:57:02.054700 31141 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:02.056051 31116 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 18:57:02.056262 31116 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474615205622-30146-0/minicluster-data/master-0/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474615205622-30146-0/minicluster-data/master-0/wal
uuid: "209fb7d1be7f44a8aecf2cd0edaeef28"
format_stamp: "Formatted at 2026-08-11 18:57:02 on dist-test-slave-h6n0"
server_key: "8d99c68be4e0365a40d4df6c7fbb8f6d"
server_key_iv: "82da1d2abd4c0c3c56adfb88686ef27e"
server_key_version: "encryptionkey@0"
I20260811 18:57:02.056478 31116 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474615205622-30146-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474615205622-30146-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474615205622-30146-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:57:02.073268 31116 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:57:02.078073 31116 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:57:02.078390 31116 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:57:02.099040 31116 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.190:38899
I20260811 18:57:02.099083 31238 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.190:38899 every 8 connection(s)
I20260811 18:57:02.100524 31116 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
I20260811 18:57:02.104506 31239 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:02.110607 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 31116
I20260811 18:57:02.110865 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474615205622-30146-0/minicluster-data/master-0/wal/instance
I20260811 18:57:02.111517 30146 external_mini_cluster.cc:1428] Setting key a7b3eca1ceca1c706afef5465591a547
I20260811 18:57:02.115033 31239 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 209fb7d1be7f44a8aecf2cd0edaeef28: Bootstrap starting.
I20260811 18:57:02.118443 31239 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 209fb7d1be7f44a8aecf2cd0edaeef28: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:02.119642 31239 log.cc:824] T 00000000000000000000000000000000 P 209fb7d1be7f44a8aecf2cd0edaeef28: Log is configured to *not* fsync() on all Append() calls
I20260811 18:57:02.122933 31239 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 209fb7d1be7f44a8aecf2cd0edaeef28: No bootstrap required, opened a new log
Aug 11 18:57:02 dist-test-slave-h6n0 krb5kdc[31094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474621, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 18:57:02.132171 31239 raft_consensus.cc:348] T 00000000000000000000000000000000 P 209fb7d1be7f44a8aecf2cd0edaeef28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "209fb7d1be7f44a8aecf2cd0edaeef28" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 38899 } }
I20260811 18:57:02.132618 31239 raft_consensus.cc:374] T 00000000000000000000000000000000 P 209fb7d1be7f44a8aecf2cd0edaeef28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:02.132732 31239 raft_consensus.cc:729] T 00000000000000000000000000000000 P 209fb7d1be7f44a8aecf2cd0edaeef28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 209fb7d1be7f44a8aecf2cd0edaeef28, State: Initialized, Role: FOLLOWER
I20260811 18:57:02.133358 31239 consensus_queue.cc:260] T 00000000000000000000000000000000 P 209fb7d1be7f44a8aecf2cd0edaeef28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "209fb7d1be7f44a8aecf2cd0edaeef28" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 38899 } }
I20260811 18:57:02.133642 31239 raft_consensus.cc:388] T 00000000000000000000000000000000 P 209fb7d1be7f44a8aecf2cd0edaeef28 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:57:02.133816 31239 raft_consensus.cc:482] T 00000000000000000000000000000000 P 209fb7d1be7f44a8aecf2cd0edaeef28 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:57:02.134070 31239 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 209fb7d1be7f44a8aecf2cd0edaeef28 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:02.137061 31239 raft_consensus.cc:504] T 00000000000000000000000000000000 P 209fb7d1be7f44a8aecf2cd0edaeef28 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "209fb7d1be7f44a8aecf2cd0edaeef28" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 38899 } }
I20260811 18:57:02.137627 31239 leader_election.cc:302] T 00000000000000000000000000000000 P 209fb7d1be7f44a8aecf2cd0edaeef28 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 209fb7d1be7f44a8aecf2cd0edaeef28; no voters:
I20260811 18:57:02.137817 31249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:02.113549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59540 (local address 127.29.112.190:38899)
0811 18:57:02.114418 (+ 869us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:02.114430 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:02.114471 (+ 41us) server_negotiation.cc:407] Connection header received
0811 18:57:02.116385 (+ 1914us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:02.116406 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:02.116769 (+ 363us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:02.117082 (+ 313us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:02.119044 (+ 1962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:02.120656 (+ 1612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:02.122431 (+ 1775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:02.123283 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:02.123418 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:02.128222 (+ 4804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:02.128230 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:02.128243 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:02.128287 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:02.132318 (+ 4031us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:02.133652 (+ 1334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:02.133664 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:02.133674 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:02.134095 (+ 421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:02.134739 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:02.134744 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:02.134750 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:02.135361 (+ 611us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:02.135606 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:02.136160 (+ 554us) server_negotiation.cc:299] Negotiation successful
0811 18:57:02.136715 (+ 555us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":654,"thread_start_us":422,"threads_started":1}
I20260811 18:57:02.138384 31239 leader_election.cc:288] T 00000000000000000000000000000000 P 209fb7d1be7f44a8aecf2cd0edaeef28 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:57:02.138510 31256 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 209fb7d1be7f44a8aecf2cd0edaeef28 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:02.139634 31256 raft_consensus.cc:686] T 00000000000000000000000000000000 P 209fb7d1be7f44a8aecf2cd0edaeef28 [term 1 LEADER]: Becoming Leader. State: Replica: 209fb7d1be7f44a8aecf2cd0edaeef28, State: Running, Role: LEADER
I20260811 18:57:02.140266 31256 consensus_queue.cc:237] T 00000000000000000000000000000000 P 209fb7d1be7f44a8aecf2cd0edaeef28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "209fb7d1be7f44a8aecf2cd0edaeef28" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 38899 } }
I20260811 18:57:02.140977 31239 sys_catalog.cc:564] T 00000000000000000000000000000000 P 209fb7d1be7f44a8aecf2cd0edaeef28 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:57:02.143463 31257 sys_catalog.cc:455] T 00000000000000000000000000000000 P 209fb7d1be7f44a8aecf2cd0edaeef28 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "209fb7d1be7f44a8aecf2cd0edaeef28" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "209fb7d1be7f44a8aecf2cd0edaeef28" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 38899 } } }
I20260811 18:57:02.143865 31257 sys_catalog.cc:458] T 00000000000000000000000000000000 P 209fb7d1be7f44a8aecf2cd0edaeef28 [sys.catalog]: This master's current role is: LEADER
I20260811 18:57:02.143575 31258 sys_catalog.cc:455] T 00000000000000000000000000000000 P 209fb7d1be7f44a8aecf2cd0edaeef28 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 209fb7d1be7f44a8aecf2cd0edaeef28. Latest consensus state: current_term: 1 leader_uuid: "209fb7d1be7f44a8aecf2cd0edaeef28" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "209fb7d1be7f44a8aecf2cd0edaeef28" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 38899 } } }
I20260811 18:57:02.143997 31258 sys_catalog.cc:458] T 00000000000000000000000000000000 P 209fb7d1be7f44a8aecf2cd0edaeef28 [sys.catalog]: This master's current role is: LEADER
I20260811 18:57:02.156211 31262 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:57:02.161298 31262 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:57:02.169625 31262 catalog_manager.cc:1347] Generated new cluster ID: 87aec34327be4550871aeaf7e67ea001
I20260811 18:57:02.169742 31262 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:57:02.184159 31262 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:57:02.185458 31262 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:57:02.198238 31262 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 209fb7d1be7f44a8aecf2cd0edaeef28: Generated new TSK 0
I20260811 18:57:02.199169 31262 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.112.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.129.keytab.
Entry for principal kudu/127.29.112.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.129.keytab.
I20260811 18:57:02.252539 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474615205622-30146-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474615205622-30146-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474615205622-30146-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474615205622-30146-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.129:0
--local_ip_for_outbound_sockets=127.29.112.129
--webserver_interface=127.29.112.129
--webserver_port=0
--tserver_master_addrs=127.29.112.190:38899
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.129.keytab
--principal=kudu/127.29.112.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:33337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:57:02.602912 31295 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:57:02.603169 31295 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:57:02.603330 31295 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:57:02.610433 31295 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:57:02.610572 31295 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:57:02.610772 31295 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.129
I20260811 18:57:02.620414 31295 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:33337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474615205622-30146-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474615205622-30146-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.129.keytab
--principal=kudu/127.29.112.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474615205622-30146-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.112.129
--webserver_port=0
--tserver_master_addrs=127.29.112.190:38899
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.129
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474615205622-30146-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:57:02.621788 31295 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:57:02.623162 31295 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:57:02.637719 31301 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:02.637741 31302 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:02.639117 31308 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:57:02.639492 31295 server_base.cc:1034] running on GCE node
I20260811 18:57:02.640121 31295 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:57:02.641111 31295 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:57:02.642426 31295 hybrid_clock.cc:648] HybridClock initialized: now 1786474622642347 us; error 66 us; skew 500 ppm
I20260811 18:57:02.642827 31295 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:57:02 dist-test-slave-h6n0 krb5kdc[31094](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474622, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:57:02.651100 31295 init.cc:377] Logged in from keytab as kudu/127.29.112.129@KRBTEST.COM (short username kudu)
I20260811 18:57:02.652796 31295 webserver.cc:460] Webserver started at http://127.29.112.129:38117/ using document root <none> and password file <none>
I20260811 18:57:02.653501 31295 fs_manager.cc:359] Metadata directory not provided
I20260811 18:57:02.653651 31295 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:57:02.653972 31295 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:57:02.657438 31295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474615205622-30146-0/minicluster-data/ts-0/data/instance:
uuid: "2bf9f52803274b3389d7e54375b42ff1"
format_stamp: "Formatted at 2026-08-11 18:57:02 on dist-test-slave-h6n0"
server_key: "23ace664d751182af3244ba8f73bc470"
server_key_iv: "2da783d328e5e45107b42253d53fe08f"
server_key_version: "encryptionkey@0"
I20260811 18:57:02.658282 31295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474615205622-30146-0/minicluster-data/ts-0/wal/instance:
uuid: "2bf9f52803274b3389d7e54375b42ff1"
format_stamp: "Formatted at 2026-08-11 18:57:02 on dist-test-slave-h6n0"
server_key: "23ace664d751182af3244ba8f73bc470"
server_key_iv: "2da783d328e5e45107b42253d53fe08f"
server_key_version: "encryptionkey@0"
I20260811 18:57:02.664002 31295 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 18:57:02.668025 31320 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:02.669466 31295 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 18:57:02.669643 31295 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474615205622-30146-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474615205622-30146-0/minicluster-data/ts-0/wal
uuid: "2bf9f52803274b3389d7e54375b42ff1"
format_stamp: "Formatted at 2026-08-11 18:57:02 on dist-test-slave-h6n0"
server_key: "23ace664d751182af3244ba8f73bc470"
server_key_iv: "2da783d328e5e45107b42253d53fe08f"
server_key_version: "encryptionkey@0"
I20260811 18:57:02.669816 31295 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474615205622-30146-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474615205622-30146-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474615205622-30146-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:57:02.709422 31295 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:57:02.714004 31295 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:57:02.714300 31295 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:57:02.715458 31295 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:57:02.716868 31295 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:57:02.716954 31295 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:02.717123 31295 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:57:02.717187 31295 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:02.763036 31295 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.129:42901
I20260811 18:57:02.763217 31515 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.129:42901 every 8 connection(s)
I20260811 18:57:02.764396 31295 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474615205622-30146-0/minicluster-data/ts-0/data/info.pb
I20260811 18:57:02.765147 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 31295
I20260811 18:57:02.765565 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474615205622-30146-0/minicluster-data/ts-0/wal/instance
I20260811 18:57:02.766816 30146 external_mini_cluster.cc:1428] Setting key 0986cc4efd7b3200d90e6182dd11ee5a
Aug 11 18:57:02 dist-test-slave-h6n0 krb5kdc[31094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474622, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.129@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 18:57:02.798909 31518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:02.768324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.129:33881 (local address 127.29.112.190:38899)
0811 18:57:02.780797 (+ 12473us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:02.780806 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:02.780834 (+ 28us) server_negotiation.cc:407] Connection header received
0811 18:57:02.781434 (+ 600us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:02.781443 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:02.781541 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:02.781696 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:02.783650 (+ 1954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:02.784962 (+ 1312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:02.787913 (+ 2951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:02.788748 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:02.788904 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:02.792835 (+ 3931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:02.792840 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:02.792843 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:02.792873 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:02.795843 (+ 2970us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:02.796564 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:02.796570 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:02.796581 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:02.796697 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:02.797144 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:02.797148 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:02.797158 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:02.797443 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:02.797604 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:02.798294 (+ 690us) server_negotiation.cc:299] Negotiation successful
0811 18:57:02.798550 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12288,"thread_start_us":12491,"threads_started":1}
I20260811 18:57:02.799961 31519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:02.768759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:38899 (local address 127.29.112.129:33881)
0811 18:57:02.779202 (+ 10443us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:02.779270 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:02.781170 (+ 1900us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:02.781953 (+ 783us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:02.781965 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:02.782647 (+ 682us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:57:02.783457 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:02.783481 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:02.785151 (+ 1670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:02.785160 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:02.787733 (+ 2573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:02.787752 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:02.789010 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:02.789020 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:02.789349 (+ 329us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:02.790256 (+ 907us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:57:02.790278 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:57:02.792585 (+ 2307us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:57:02.795990 (+ 3405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:02.795999 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:02.796015 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:02.796433 (+ 418us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:02.796812 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:02.796816 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:02.796828 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:02.797017 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:02.797581 (+ 564us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:57:02.797589 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:57:02.797909 (+ 320us) client_negotiation.cc:769] Sending connection context
0811 18:57:02.798169 (+ 260us) client_negotiation.cc:240] Negotiation successful
0811 18:57:02.798705 (+ 536us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10133,"thread_start_us":9909,"threads_started":1}
I20260811 18:57:02.801466 31516 heartbeater.cc:347] Connected to a master server at 127.29.112.190:38899
I20260811 18:57:02.801744 31516 heartbeater.cc:464] Registering TS with master...
I20260811 18:57:02.802474 31516 heartbeater.cc:511] Master 127.29.112.190:38899 requested a full tablet report, sending...
I20260811 18:57:02.804515 31172 ts_manager.cc:194] Registered new tserver with Master: 2bf9f52803274b3389d7e54375b42ff1 (127.29.112.129:42901)
I20260811 18:57:02.805934 31172 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.129@KRBTEST.COM'} at 127.29.112.129:33881
WARNING: no policy specified for kudu/127.29.112.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.130.keytab.
Entry for principal kudu/127.29.112.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.130.keytab.
I20260811 18:57:02.838056 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474615205622-30146-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474615205622-30146-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474615205622-30146-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474615205622-30146-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.130:0
--local_ip_for_outbound_sockets=127.29.112.130
--webserver_interface=127.29.112.130
--webserver_port=0
--tserver_master_addrs=127.29.112.190:38899
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.130.keytab
--principal=kudu/127.29.112.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:33337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:57:03.208014 31523 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:57:03.208331 31523 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:57:03.208451 31523 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:57:03.216461 31523 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:57:03.216569 31523 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:57:03.216681 31523 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.130
I20260811 18:57:03.226924 31523 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:33337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474615205622-30146-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474615205622-30146-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.130.keytab
--principal=kudu/127.29.112.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474615205622-30146-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.112.130
--webserver_port=0
--tserver_master_addrs=127.29.112.190:38899
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.130
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474615205622-30146-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:57:03.228158 31523 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:57:03.229424 31523 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:57:03.242624 31534 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:03.242736 31529 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:03.243109 31530 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:57:03.244056 31523 server_base.cc:1034] running on GCE node
I20260811 18:57:03.244684 31523 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:57:03.245770 31523 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:57:03.247006 31523 hybrid_clock.cc:648] HybridClock initialized: now 1786474623246950 us; error 105 us; skew 500 ppm
I20260811 18:57:03.247344 31523 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:57:03 dist-test-slave-h6n0 krb5kdc[31094](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474623, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:57:03.250702 31523 init.cc:377] Logged in from keytab as kudu/127.29.112.130@KRBTEST.COM (short username kudu)
I20260811 18:57:03.252177 31523 webserver.cc:460] Webserver started at http://127.29.112.130:46229/ using document root <none> and password file <none>
I20260811 18:57:03.252863 31523 fs_manager.cc:359] Metadata directory not provided
I20260811 18:57:03.253021 31523 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:57:03.253355 31523 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:57:03.256481 31523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474615205622-30146-0/minicluster-data/ts-1/data/instance:
uuid: "1ff75090686a4259a435c8ba9a041092"
format_stamp: "Formatted at 2026-08-11 18:57:03 on dist-test-slave-h6n0"
server_key: "8520dc36f8c64557b53449693a4896ab"
server_key_iv: "d775901e1818c9c96496a13098f9466d"
server_key_version: "encryptionkey@0"
I20260811 18:57:03.257303 31523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474615205622-30146-0/minicluster-data/ts-1/wal/instance:
uuid: "1ff75090686a4259a435c8ba9a041092"
format_stamp: "Formatted at 2026-08-11 18:57:03 on dist-test-slave-h6n0"
server_key: "8520dc36f8c64557b53449693a4896ab"
server_key_iv: "d775901e1818c9c96496a13098f9466d"
server_key_version: "encryptionkey@0"
I20260811 18:57:03.263844 31523 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260811 18:57:03.267496 31543 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:03.268731 31523 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 18:57:03.268914 31523 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474615205622-30146-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474615205622-30146-0/minicluster-data/ts-1/wal
uuid: "1ff75090686a4259a435c8ba9a041092"
format_stamp: "Formatted at 2026-08-11 18:57:03 on dist-test-slave-h6n0"
server_key: "8520dc36f8c64557b53449693a4896ab"
server_key_iv: "d775901e1818c9c96496a13098f9466d"
server_key_version: "encryptionkey@0"
I20260811 18:57:03.269117 31523 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474615205622-30146-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474615205622-30146-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474615205622-30146-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:57:03.291178 31523 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:57:03.295641 31523 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:57:03.295950 31523 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:57:03.297004 31523 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:57:03.298378 31523 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:57:03.298506 31523 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:03.298703 31523 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:57:03.298780 31523 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:03.337111 31523 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.130:33733
I20260811 18:57:03.337208 31763 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.130:33733 every 8 connection(s)
I20260811 18:57:03.338404 31523 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474615205622-30146-0/minicluster-data/ts-1/data/info.pb
I20260811 18:57:03.344127 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 31523
I20260811 18:57:03.344336 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474615205622-30146-0/minicluster-data/ts-1/wal/instance
I20260811 18:57:03.344786 30146 external_mini_cluster.cc:1428] Setting key af0af61cd2ec6f7d9f1e63431062bc81
Aug 11 18:57:03 dist-test-slave-h6n0 krb5kdc[31094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474623, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.130@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 18:57:03.358825 31766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:03.340979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.130:51753 (local address 127.29.112.190:38899)
0811 18:57:03.341401 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:03.341405 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:03.342464 (+ 1059us) server_negotiation.cc:407] Connection header received
0811 18:57:03.344327 (+ 1863us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:03.344333 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:03.344421 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:03.344545 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:03.346020 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:03.346895 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:03.348423 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:03.348843 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:03.348976 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:03.352875 (+ 3899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:03.352880 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:03.352883 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:03.352914 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:03.355663 (+ 2749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:03.356399 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:03.356404 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:03.356407 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:03.356554 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:03.356977 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:03.356983 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:03.356987 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:03.357298 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:03.357498 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:03.358168 (+ 670us) server_negotiation.cc:299] Negotiation successful
0811 18:57:03.358388 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":40,"server-negotiator.queue_time_us":317,"thread_start_us":202,"threads_started":1}
I20260811 18:57:03.359777 31767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:03.341533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:38899 (local address 127.29.112.130:51753)
0811 18:57:03.342313 (+ 780us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:03.342361 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:03.344077 (+ 1716us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:03.344763 (+ 686us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:03.344770 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:03.345314 (+ 544us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:57:03.345850 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:03.345867 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:03.347047 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:03.347058 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:03.348267 (+ 1209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:03.348279 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:03.349104 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:03.349114 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:03.349317 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:03.350089 (+ 772us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:57:03.350116 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:57:03.352645 (+ 2529us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:57:03.355805 (+ 3160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:03.355814 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:03.355828 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:03.356245 (+ 417us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:03.356667 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:03.356672 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:03.356676 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:03.356864 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:03.357468 (+ 604us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:57:03.357476 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:57:03.357813 (+ 337us) client_negotiation.cc:769] Sending connection context
0811 18:57:03.358093 (+ 280us) client_negotiation.cc:240] Negotiation successful
0811 18:57:03.358645 (+ 552us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":570,"thread_start_us":284,"threads_started":1}
I20260811 18:57:03.361228 31764 heartbeater.cc:347] Connected to a master server at 127.29.112.190:38899
I20260811 18:57:03.361531 31764 heartbeater.cc:464] Registering TS with master...
I20260811 18:57:03.362226 31764 heartbeater.cc:511] Master 127.29.112.190:38899 requested a full tablet report, sending...
I20260811 18:57:03.363778 31172 ts_manager.cc:194] Registered new tserver with Master: 1ff75090686a4259a435c8ba9a041092 (127.29.112.130:33733)
I20260811 18:57:03.365025 31172 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.130@KRBTEST.COM'} at 127.29.112.130:51753
WARNING: no policy specified for kudu/127.29.112.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.131.keytab.
Entry for principal kudu/127.29.112.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.131.keytab.
I20260811 18:57:03.391746 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474615205622-30146-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474615205622-30146-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474615205622-30146-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474615205622-30146-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.131:0
--local_ip_for_outbound_sockets=127.29.112.131
--webserver_interface=127.29.112.131
--webserver_port=0
--tserver_master_addrs=127.29.112.190:38899
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.131.keytab
--principal=kudu/127.29.112.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:33337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:57:03.737545 31776 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:57:03.737829 31776 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:57:03.737928 31776 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:57:03.745452 31776 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:57:03.745591 31776 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:57:03.745716 31776 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.131
I20260811 18:57:03.755787 31776 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:33337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474615205622-30146-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474615205622-30146-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.131.keytab
--principal=kudu/127.29.112.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474615205622-30146-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.112.131
--webserver_port=0
--tserver_master_addrs=127.29.112.190:38899
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.131
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474615205622-30146-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:57:03.757093 31776 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:57:03.758373 31776 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:57:03.772310 31784 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:03.772384 31785 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:03.772581 31792 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:57:03.773556 31776 server_base.cc:1034] running on GCE node
I20260811 18:57:03.774281 31776 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:57:03.775588 31776 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:57:03.776901 31776 hybrid_clock.cc:648] HybridClock initialized: now 1786474623776788 us; error 90 us; skew 500 ppm
I20260811 18:57:03.777431 31776 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:57:03 dist-test-slave-h6n0 krb5kdc[31094](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474623, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:57:03.782117 31776 init.cc:377] Logged in from keytab as kudu/127.29.112.131@KRBTEST.COM (short username kudu)
I20260811 18:57:03.784488 31776 webserver.cc:460] Webserver started at http://127.29.112.131:33961/ using document root <none> and password file <none>
I20260811 18:57:03.785734 31776 fs_manager.cc:359] Metadata directory not provided
I20260811 18:57:03.785925 31776 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:57:03.786353 31776 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:57:03.789517 31776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474615205622-30146-0/minicluster-data/ts-2/data/instance:
uuid: "b108b7dddb8d40838456b53ca0eca6d7"
format_stamp: "Formatted at 2026-08-11 18:57:03 on dist-test-slave-h6n0"
server_key: "c9e6623eeb555b1ff83bee4f91daf3fa"
server_key_iv: "cd63e33175f688b76afb807b592fba5f"
server_key_version: "encryptionkey@0"
I20260811 18:57:03.790336 31776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474615205622-30146-0/minicluster-data/ts-2/wal/instance:
uuid: "b108b7dddb8d40838456b53ca0eca6d7"
format_stamp: "Formatted at 2026-08-11 18:57:03 on dist-test-slave-h6n0"
server_key: "c9e6623eeb555b1ff83bee4f91daf3fa"
server_key_iv: "cd63e33175f688b76afb807b592fba5f"
server_key_version: "encryptionkey@0"
I20260811 18:57:03.796018 31776 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260811 18:57:03.800076 31803 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:03.802124 31776 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.002s
I20260811 18:57:03.802412 31776 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474615205622-30146-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474615205622-30146-0/minicluster-data/ts-2/wal
uuid: "b108b7dddb8d40838456b53ca0eca6d7"
format_stamp: "Formatted at 2026-08-11 18:57:03 on dist-test-slave-h6n0"
server_key: "c9e6623eeb555b1ff83bee4f91daf3fa"
server_key_iv: "cd63e33175f688b76afb807b592fba5f"
server_key_version: "encryptionkey@0"
I20260811 18:57:03.802827 31776 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474615205622-30146-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474615205622-30146-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474615205622-30146-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:57:03.808920 31516 heartbeater.cc:503] Master 127.29.112.190:38899 was elected leader, sending a full tablet report...
I20260811 18:57:03.837431 31776 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:57:03.841852 31776 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:57:03.842111 31776 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:57:03.843251 31776 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:57:03.844563 31776 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:57:03.844637 31776 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:03.844734 31776 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:57:03.844779 31776 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:03.883536 31776 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.131:46677
I20260811 18:57:03.883755 32006 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.131:46677 every 8 connection(s)
I20260811 18:57:03.884948 31776 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474615205622-30146-0/minicluster-data/ts-2/data/info.pb
I20260811 18:57:03.894299 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 31776
I20260811 18:57:03.894601 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474615205622-30146-0/minicluster-data/ts-2/wal/instance
I20260811 18:57:03.895069 30146 external_mini_cluster.cc:1428] Setting key e3cc4814c17f7135d211c465bbf0d9d0
Aug 11 18:57:03 dist-test-slave-h6n0 krb5kdc[31094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474623, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.131@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 18:57:03.910236 32012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:03.887650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.131:57763 (local address 127.29.112.190:38899)
0811 18:57:03.888105 (+ 455us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:03.888110 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:03.889176 (+ 1066us) server_negotiation.cc:407] Connection header received
0811 18:57:03.891118 (+ 1942us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:03.891124 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:03.891221 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:03.891341 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:03.892928 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:03.894130 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:03.896937 (+ 2807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:03.897576 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:03.897717 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:03.902712 (+ 4995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:03.902718 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:03.902721 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:03.902758 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:03.905462 (+ 2704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:03.906630 (+ 1168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:03.906637 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:03.906639 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:03.906941 (+ 302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:03.907751 (+ 810us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:03.907756 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:03.907758 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:03.908187 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:03.908365 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:03.909451 (+ 1086us) server_negotiation.cc:299] Negotiation successful
0811 18:57:03.909743 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":215,"threads_started":1}
I20260811 18:57:03.910945 32013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:03.888194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:38899 (local address 127.29.112.131:57763)
0811 18:57:03.888918 (+ 724us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:03.888956 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:03.890835 (+ 1879us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:03.891595 (+ 760us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:03.891603 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:03.892163 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:57:03.892751 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:03.892768 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:03.894512 (+ 1744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:03.894520 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:03.896706 (+ 2186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:03.896730 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:03.897940 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:03.897951 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:03.898501 (+ 550us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:03.899463 (+ 962us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:57:03.899500 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:57:03.902369 (+ 2869us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:57:03.905659 (+ 3290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:03.905668 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:03.905684 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:03.906282 (+ 598us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:03.907179 (+ 897us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:03.907186 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:03.907189 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:03.907554 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:03.908393 (+ 839us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:57:03.908400 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:57:03.908794 (+ 394us) client_negotiation.cc:769] Sending connection context
0811 18:57:03.909069 (+ 275us) client_negotiation.cc:240] Negotiation successful
0811 18:57:03.909521 (+ 452us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":315,"threads_started":1}
I20260811 18:57:03.912573 32008 heartbeater.cc:347] Connected to a master server at 127.29.112.190:38899
I20260811 18:57:03.912943 32008 heartbeater.cc:464] Registering TS with master...
I20260811 18:57:03.913791 32008 heartbeater.cc:511] Master 127.29.112.190:38899 requested a full tablet report, sending...
I20260811 18:57:03.915488 31171 ts_manager.cc:194] Registered new tserver with Master: b108b7dddb8d40838456b53ca0eca6d7 (127.29.112.131:46677)
I20260811 18:57:03.916419 31171 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.131@KRBTEST.COM'} at 127.29.112.131:57763
I20260811 18:57:03.925457 30146 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 18:57:03.946578 32012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:03.931811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59550 (local address 127.29.112.190:38899)
0811 18:57:03.932017 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:03.932023 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:03.932446 (+ 423us) server_negotiation.cc:407] Connection header received
0811 18:57:03.932643 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:03.932649 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:03.932749 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:03.932857 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:03.934389 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:03.935175 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:03.936417 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:03.936837 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:03.936968 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:03.939505 (+ 2537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:03.939511 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:03.939514 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:03.939544 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:03.943255 (+ 3711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:03.944284 (+ 1029us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:03.944290 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:03.944294 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:03.944475 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:03.944930 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:03.944938 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:03.944942 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:03.945301 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:03.945612 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:03.945958 (+ 346us) server_negotiation.cc:299] Negotiation successful
0811 18:57:03.946205 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 18:57:03.953526 31171 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59550:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:03.960956 31171 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:03.989069 32025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:03.976263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.131:46677 (local address 127.0.0.1:53814)
0811 18:57:03.977283 (+ 1020us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:03.977321 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:03.977530 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:03.978630 (+ 1100us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:03.978639 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:03.978676 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:57:03.979070 (+ 394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:03.979086 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:03.981305 (+ 2219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:03.981313 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:03.984726 (+ 3413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:03.984798 (+ 72us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:03.987715 (+ 2917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:03.987724 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:03.988013 (+ 289us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:03.988105 (+ 92us) client_negotiation.cc:769] Sending connection context
0811 18:57:03.988278 (+ 173us) client_negotiation.cc:240] Negotiation successful
0811 18:57:03.988572 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":665,"mutex_wait_us":66,"thread_start_us":360,"threads_started":1}
I20260811 18:57:03.990397 32026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:03.976507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53814 (local address 127.29.112.131:46677)
0811 18:57:03.977895 (+ 1388us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:03.977905 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:03.977941 (+ 36us) server_negotiation.cc:407] Connection header received
0811 18:57:03.978055 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:03.978066 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:03.978340 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:03.978588 (+ 248us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:57:03.979303 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:03.981080 (+ 1777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:03.985071 (+ 3991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:03.987022 (+ 1951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:03.987271 (+ 249us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:03.987528 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:03.989722 (+ 2194us) server_negotiation.cc:299] Negotiation successful
0811 18:57:03.989907 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1204,"thread_start_us":1014,"threads_started":1}
I20260811 18:57:03.991336 32029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:03.977118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36408 (local address 127.29.112.130:33733)
0811 18:57:03.979296 (+ 2178us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:03.979304 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:03.979330 (+ 26us) server_negotiation.cc:407] Connection header received
0811 18:57:03.979422 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:03.979429 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:03.979672 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:03.979858 (+ 186us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:57:03.980977 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:03.983030 (+ 2053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:03.986628 (+ 3598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:03.988736 (+ 2108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:03.988948 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:03.989165 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:03.990632 (+ 1467us) server_negotiation.cc:299] Negotiation successful
0811 18:57:03.990867 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2002,"thread_start_us":1843,"threads_started":1}
I20260811 18:57:03.991624 32030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:03.977841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.130:33733 (local address 127.0.0.1:36408)
0811 18:57:03.978615 (+ 774us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:03.978641 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:03.978822 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:03.980139 (+ 1317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:03.980147 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:03.980183 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:57:03.980744 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:03.980759 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:03.983238 (+ 2479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:03.983249 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:03.986341 (+ 3092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:03.986363 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:03.989342 (+ 2979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:03.989347 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:03.989595 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:03.989651 (+ 56us) client_negotiation.cc:769] Sending connection context
0811 18:57:03.989769 (+ 118us) client_negotiation.cc:240] Negotiation successful
0811 18:57:03.989969 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":583,"mutex_wait_us":28,"spinlock_wait_cycles":15232,"thread_start_us":368,"threads_started":1}
I20260811 18:57:03.995500 31642 tablet_service.cc:1467] Processing CreateTablet for tablet dfdb2e11fae24836b81ebb0673610f00 (DEFAULT_TABLE table=test-table [id=a80f199bc1c94d40ab948cca06176b87]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:03.995708 31895 tablet_service.cc:1467] Processing CreateTablet for tablet dfdb2e11fae24836b81ebb0673610f00 (DEFAULT_TABLE table=test-table [id=a80f199bc1c94d40ab948cca06176b87]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:03.996963 31642 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dfdb2e11fae24836b81ebb0673610f00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:03.997164 31895 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dfdb2e11fae24836b81ebb0673610f00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:04.002390 32025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:03.979923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:42901 (local address 127.0.0.1:49576)
0811 18:57:03.989426 (+ 9503us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:03.989448 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:03.989543 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:03.995760 (+ 6217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:03.995765 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:03.995793 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:57:03.995953 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:03.995962 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:03.997878 (+ 1916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:03.997884 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:03.999776 (+ 1892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:03.999794 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:04.001585 (+ 1791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:04.001593 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:04.001780 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:04.001820 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 18:57:04.001928 (+ 108us) client_negotiation.cc:240] Negotiation successful
0811 18:57:04.002069 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9303,"thread_start_us":12415,"threads_started":1}
I20260811 18:57:04.002804 32032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:03.980274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49576 (local address 127.29.112.129:42901)
0811 18:57:03.995152 (+ 14878us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:03.995160 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:03.995191 (+ 31us) server_negotiation.cc:407] Connection header received
0811 18:57:03.995303 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:03.995310 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:03.995538 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:03.995756 (+ 218us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:57:03.996078 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:03.997619 (+ 1541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:03.999962 (+ 2343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:04.001395 (+ 1433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:04.001605 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:04.001768 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:04.002222 (+ 454us) server_negotiation.cc:299] Negotiation successful
0811 18:57:04.002406 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14708,"thread_start_us":14407,"threads_started":1}
I20260811 18:57:04.006126 31398 tablet_service.cc:1467] Processing CreateTablet for tablet dfdb2e11fae24836b81ebb0673610f00 (DEFAULT_TABLE table=test-table [id=a80f199bc1c94d40ab948cca06176b87]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:04.007685 31398 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dfdb2e11fae24836b81ebb0673610f00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:04.011190 32033 tablet_bootstrap.cc:492] T dfdb2e11fae24836b81ebb0673610f00 P b108b7dddb8d40838456b53ca0eca6d7: Bootstrap starting.
I20260811 18:57:04.014143 32034 tablet_bootstrap.cc:492] T dfdb2e11fae24836b81ebb0673610f00 P 1ff75090686a4259a435c8ba9a041092: Bootstrap starting.
I20260811 18:57:04.015051 32033 tablet_bootstrap.cc:654] T dfdb2e11fae24836b81ebb0673610f00 P b108b7dddb8d40838456b53ca0eca6d7: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:04.016431 32033 log.cc:824] T dfdb2e11fae24836b81ebb0673610f00 P b108b7dddb8d40838456b53ca0eca6d7: Log is configured to *not* fsync() on all Append() calls
I20260811 18:57:04.018020 32034 tablet_bootstrap.cc:654] T dfdb2e11fae24836b81ebb0673610f00 P 1ff75090686a4259a435c8ba9a041092: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:04.019264 32034 log.cc:824] T dfdb2e11fae24836b81ebb0673610f00 P 1ff75090686a4259a435c8ba9a041092: Log is configured to *not* fsync() on all Append() calls
I20260811 18:57:04.020074 32033 tablet_bootstrap.cc:492] T dfdb2e11fae24836b81ebb0673610f00 P b108b7dddb8d40838456b53ca0eca6d7: No bootstrap required, opened a new log
I20260811 18:57:04.020355 32033 ts_tablet_manager.cc:1397] T dfdb2e11fae24836b81ebb0673610f00 P b108b7dddb8d40838456b53ca0eca6d7: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.003s
I20260811 18:57:04.021512 32041 tablet_bootstrap.cc:492] T dfdb2e11fae24836b81ebb0673610f00 P 2bf9f52803274b3389d7e54375b42ff1: Bootstrap starting.
I20260811 18:57:04.022394 32034 tablet_bootstrap.cc:492] T dfdb2e11fae24836b81ebb0673610f00 P 1ff75090686a4259a435c8ba9a041092: No bootstrap required, opened a new log
I20260811 18:57:04.022680 32034 ts_tablet_manager.cc:1397] T dfdb2e11fae24836b81ebb0673610f00 P 1ff75090686a4259a435c8ba9a041092: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260811 18:57:04.024507 32041 tablet_bootstrap.cc:654] T dfdb2e11fae24836b81ebb0673610f00 P 2bf9f52803274b3389d7e54375b42ff1: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:04.025610 32041 log.cc:824] T dfdb2e11fae24836b81ebb0673610f00 P 2bf9f52803274b3389d7e54375b42ff1: Log is configured to *not* fsync() on all Append() calls
I20260811 18:57:04.029155 32041 tablet_bootstrap.cc:492] T dfdb2e11fae24836b81ebb0673610f00 P 2bf9f52803274b3389d7e54375b42ff1: No bootstrap required, opened a new log
I20260811 18:57:04.029428 32041 ts_tablet_manager.cc:1397] T dfdb2e11fae24836b81ebb0673610f00 P 2bf9f52803274b3389d7e54375b42ff1: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260811 18:57:04.031170 32033 raft_consensus.cc:348] T dfdb2e11fae24836b81ebb0673610f00 P b108b7dddb8d40838456b53ca0eca6d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b108b7dddb8d40838456b53ca0eca6d7" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 46677 } } peers { permanent_uuid: "1ff75090686a4259a435c8ba9a041092" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33733 } } peers { permanent_uuid: "2bf9f52803274b3389d7e54375b42ff1" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 42901 } }
I20260811 18:57:04.031632 32033 raft_consensus.cc:374] T dfdb2e11fae24836b81ebb0673610f00 P b108b7dddb8d40838456b53ca0eca6d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:04.031744 32033 raft_consensus.cc:729] T dfdb2e11fae24836b81ebb0673610f00 P b108b7dddb8d40838456b53ca0eca6d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b108b7dddb8d40838456b53ca0eca6d7, State: Initialized, Role: FOLLOWER
I20260811 18:57:04.032368 32033 consensus_queue.cc:260] T dfdb2e11fae24836b81ebb0673610f00 P b108b7dddb8d40838456b53ca0eca6d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b108b7dddb8d40838456b53ca0eca6d7" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 46677 } } peers { permanent_uuid: "1ff75090686a4259a435c8ba9a041092" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33733 } } peers { permanent_uuid: "2bf9f52803274b3389d7e54375b42ff1" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 42901 } }
I20260811 18:57:04.033496 32008 heartbeater.cc:503] Master 127.29.112.190:38899 was elected leader, sending a full tablet report...
I20260811 18:57:04.033277 32034 raft_consensus.cc:348] T dfdb2e11fae24836b81ebb0673610f00 P 1ff75090686a4259a435c8ba9a041092 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b108b7dddb8d40838456b53ca0eca6d7" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 46677 } } peers { permanent_uuid: "1ff75090686a4259a435c8ba9a041092" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33733 } } peers { permanent_uuid: "2bf9f52803274b3389d7e54375b42ff1" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 42901 } }
I20260811 18:57:04.033696 32034 raft_consensus.cc:374] T dfdb2e11fae24836b81ebb0673610f00 P 1ff75090686a4259a435c8ba9a041092 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:04.033787 32034 raft_consensus.cc:729] T dfdb2e11fae24836b81ebb0673610f00 P 1ff75090686a4259a435c8ba9a041092 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ff75090686a4259a435c8ba9a041092, State: Initialized, Role: FOLLOWER
I20260811 18:57:04.034430 32034 consensus_queue.cc:260] T dfdb2e11fae24836b81ebb0673610f00 P 1ff75090686a4259a435c8ba9a041092 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b108b7dddb8d40838456b53ca0eca6d7" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 46677 } } peers { permanent_uuid: "1ff75090686a4259a435c8ba9a041092" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33733 } } peers { permanent_uuid: "2bf9f52803274b3389d7e54375b42ff1" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 42901 } }
I20260811 18:57:04.034750 32033 ts_tablet_manager.cc:1428] T dfdb2e11fae24836b81ebb0673610f00 P b108b7dddb8d40838456b53ca0eca6d7: Time spent starting tablet: real 0.014s user 0.012s sys 0.004s
I20260811 18:57:04.036165 31764 heartbeater.cc:503] Master 127.29.112.190:38899 was elected leader, sending a full tablet report...
I20260811 18:57:04.036645 32034 ts_tablet_manager.cc:1428] T dfdb2e11fae24836b81ebb0673610f00 P 1ff75090686a4259a435c8ba9a041092: Time spent starting tablet: real 0.014s user 0.009s sys 0.004s
I20260811 18:57:04.039880 32041 raft_consensus.cc:348] T dfdb2e11fae24836b81ebb0673610f00 P 2bf9f52803274b3389d7e54375b42ff1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b108b7dddb8d40838456b53ca0eca6d7" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 46677 } } peers { permanent_uuid: "1ff75090686a4259a435c8ba9a041092" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33733 } } peers { permanent_uuid: "2bf9f52803274b3389d7e54375b42ff1" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 42901 } }
I20260811 18:57:04.040277 32041 raft_consensus.cc:374] T dfdb2e11fae24836b81ebb0673610f00 P 2bf9f52803274b3389d7e54375b42ff1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:04.040396 32041 raft_consensus.cc:729] T dfdb2e11fae24836b81ebb0673610f00 P 2bf9f52803274b3389d7e54375b42ff1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2bf9f52803274b3389d7e54375b42ff1, State: Initialized, Role: FOLLOWER
I20260811 18:57:04.041145 32041 consensus_queue.cc:260] T dfdb2e11fae24836b81ebb0673610f00 P 2bf9f52803274b3389d7e54375b42ff1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b108b7dddb8d40838456b53ca0eca6d7" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 46677 } } peers { permanent_uuid: "1ff75090686a4259a435c8ba9a041092" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33733 } } peers { permanent_uuid: "2bf9f52803274b3389d7e54375b42ff1" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 42901 } }
I20260811 18:57:04.042817 32041 ts_tablet_manager.cc:1428] T dfdb2e11fae24836b81ebb0673610f00 P 2bf9f52803274b3389d7e54375b42ff1: Time spent starting tablet: real 0.013s user 0.005s sys 0.008s
W20260811 18:57:04.090497 31765 tablet.cc:2369] T dfdb2e11fae24836b81ebb0673610f00 P 1ff75090686a4259a435c8ba9a041092: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 18:57:04.136794 32010 tablet.cc:2369] T dfdb2e11fae24836b81ebb0673610f00 P b108b7dddb8d40838456b53ca0eca6d7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:57:04.154178 32046 raft_consensus.cc:482] T dfdb2e11fae24836b81ebb0673610f00 P 2bf9f52803274b3389d7e54375b42ff1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:04.154592 32046 raft_consensus.cc:504] T dfdb2e11fae24836b81ebb0673610f00 P 2bf9f52803274b3389d7e54375b42ff1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b108b7dddb8d40838456b53ca0eca6d7" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 46677 } } peers { permanent_uuid: "1ff75090686a4259a435c8ba9a041092" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33733 } } peers { permanent_uuid: "2bf9f52803274b3389d7e54375b42ff1" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 42901 } }
I20260811 18:57:04.156555 32046 leader_election.cc:288] T dfdb2e11fae24836b81ebb0673610f00 P 2bf9f52803274b3389d7e54375b42ff1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b108b7dddb8d40838456b53ca0eca6d7 (127.29.112.131:46677), 1ff75090686a4259a435c8ba9a041092 (127.29.112.130:33733)
I20260811 18:57:04.164394 32029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:04.157292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.129:54893 (local address 127.29.112.130:33733)
0811 18:57:04.157608 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:04.157616 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:04.158244 (+ 628us) server_negotiation.cc:407] Connection header received
0811 18:57:04.158632 (+ 388us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:04.158639 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:04.158724 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:04.158817 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:57:04.159535 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:04.160351 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:04.162476 (+ 2125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:04.163356 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:04.163480 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:04.163572 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:04.163969 (+ 397us) server_negotiation.cc:299] Negotiation successful
0811 18:57:04.164119 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260811 18:57:04.164404 32026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:04.157325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.129:56975 (local address 127.29.112.131:46677)
0811 18:57:04.157775 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:04.157783 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:04.158600 (+ 817us) server_negotiation.cc:407] Connection header received
0811 18:57:04.158835 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:04.158840 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:04.158919 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:04.159023 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:57:04.159680 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:04.160645 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:04.162648 (+ 2003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:04.163452 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:04.163613 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:04.163694 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:04.164019 (+ 325us) server_negotiation.cc:299] Negotiation successful
0811 18:57:04.164139 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316}
I20260811 18:57:04.164561 32049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:04.157054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.130:33733 (local address 127.29.112.129:54893)
0811 18:57:04.158107 (+ 1053us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:04.158131 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:04.158306 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:04.158853 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:04.158858 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:04.158901 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:57:04.159367 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:04.159385 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:04.160486 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:04.160491 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:04.162298 (+ 1807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:04.162311 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:04.163588 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:04.163596 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:04.163793 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:04.163818 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 18:57:04.163903 (+ 85us) client_negotiation.cc:240] Negotiation successful
0811 18:57:04.164073 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":872,"mutex_wait_us":1,"spinlock_wait_cycles":25984,"thread_start_us":910,"threads_started":1}
I20260811 18:57:04.164624 32047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:04.157054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.131:46677 (local address 127.29.112.129:56975)
0811 18:57:04.158339 (+ 1285us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:04.158354 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:04.158668 (+ 314us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:04.159060 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:04.159063 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:04.159094 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:57:04.159522 (+ 428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:04.159534 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:04.160840 (+ 1306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:04.160844 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:04.162464 (+ 1620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:04.162480 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:04.163704 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:04.163708 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:04.163830 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:04.163853 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 18:57:04.163969 (+ 116us) client_negotiation.cc:240] Negotiation successful
0811 18:57:04.164145 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1123,"mutex_wait_us":108,"thread_start_us":446,"threads_started":1}
I20260811 18:57:04.165836 31933 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfdb2e11fae24836b81ebb0673610f00" candidate_uuid: "2bf9f52803274b3389d7e54375b42ff1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b108b7dddb8d40838456b53ca0eca6d7" is_pre_election: true
I20260811 18:57:04.165874 31676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfdb2e11fae24836b81ebb0673610f00" candidate_uuid: "2bf9f52803274b3389d7e54375b42ff1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ff75090686a4259a435c8ba9a041092" is_pre_election: true
I20260811 18:57:04.166476 31676 raft_consensus.cc:2454] T dfdb2e11fae24836b81ebb0673610f00 P 1ff75090686a4259a435c8ba9a041092 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2bf9f52803274b3389d7e54375b42ff1 in term 0.
I20260811 18:57:04.166522 31933 raft_consensus.cc:2454] T dfdb2e11fae24836b81ebb0673610f00 P b108b7dddb8d40838456b53ca0eca6d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2bf9f52803274b3389d7e54375b42ff1 in term 0.
I20260811 18:57:04.167315 31326 leader_election.cc:302] T dfdb2e11fae24836b81ebb0673610f00 P 2bf9f52803274b3389d7e54375b42ff1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1ff75090686a4259a435c8ba9a041092, 2bf9f52803274b3389d7e54375b42ff1, b108b7dddb8d40838456b53ca0eca6d7; no voters:
I20260811 18:57:04.167766 32046 raft_consensus.cc:2781] T dfdb2e11fae24836b81ebb0673610f00 P 2bf9f52803274b3389d7e54375b42ff1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:04.167963 32046 raft_consensus.cc:482] T dfdb2e11fae24836b81ebb0673610f00 P 2bf9f52803274b3389d7e54375b42ff1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:04.168154 32046 raft_consensus.cc:3037] T dfdb2e11fae24836b81ebb0673610f00 P 2bf9f52803274b3389d7e54375b42ff1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:04.170922 32046 raft_consensus.cc:504] T dfdb2e11fae24836b81ebb0673610f00 P 2bf9f52803274b3389d7e54375b42ff1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b108b7dddb8d40838456b53ca0eca6d7" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 46677 } } peers { permanent_uuid: "1ff75090686a4259a435c8ba9a041092" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33733 } } peers { permanent_uuid: "2bf9f52803274b3389d7e54375b42ff1" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 42901 } }
I20260811 18:57:04.171657 32046 leader_election.cc:288] T dfdb2e11fae24836b81ebb0673610f00 P 2bf9f52803274b3389d7e54375b42ff1 [CANDIDATE]: Term 1 election: Requested vote from peers b108b7dddb8d40838456b53ca0eca6d7 (127.29.112.131:46677), 1ff75090686a4259a435c8ba9a041092 (127.29.112.130:33733)
I20260811 18:57:04.172565 31676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfdb2e11fae24836b81ebb0673610f00" candidate_uuid: "2bf9f52803274b3389d7e54375b42ff1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ff75090686a4259a435c8ba9a041092"
I20260811 18:57:04.172552 31933 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfdb2e11fae24836b81ebb0673610f00" candidate_uuid: "2bf9f52803274b3389d7e54375b42ff1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b108b7dddb8d40838456b53ca0eca6d7"
I20260811 18:57:04.172835 31933 raft_consensus.cc:3037] T dfdb2e11fae24836b81ebb0673610f00 P b108b7dddb8d40838456b53ca0eca6d7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:04.172860 31676 raft_consensus.cc:3037] T dfdb2e11fae24836b81ebb0673610f00 P 1ff75090686a4259a435c8ba9a041092 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:04.176717 31933 raft_consensus.cc:2454] T dfdb2e11fae24836b81ebb0673610f00 P b108b7dddb8d40838456b53ca0eca6d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2bf9f52803274b3389d7e54375b42ff1 in term 1.
I20260811 18:57:04.176710 31676 raft_consensus.cc:2454] T dfdb2e11fae24836b81ebb0673610f00 P 1ff75090686a4259a435c8ba9a041092 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2bf9f52803274b3389d7e54375b42ff1 in term 1.
I20260811 18:57:04.177268 31326 leader_election.cc:302] T dfdb2e11fae24836b81ebb0673610f00 P 2bf9f52803274b3389d7e54375b42ff1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1ff75090686a4259a435c8ba9a041092, 2bf9f52803274b3389d7e54375b42ff1, b108b7dddb8d40838456b53ca0eca6d7; no voters:
I20260811 18:57:04.177641 32046 raft_consensus.cc:2781] T dfdb2e11fae24836b81ebb0673610f00 P 2bf9f52803274b3389d7e54375b42ff1 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:04.178154 32046 raft_consensus.cc:686] T dfdb2e11fae24836b81ebb0673610f00 P 2bf9f52803274b3389d7e54375b42ff1 [term 1 LEADER]: Becoming Leader. State: Replica: 2bf9f52803274b3389d7e54375b42ff1, State: Running, Role: LEADER
I20260811 18:57:04.178768 32046 consensus_queue.cc:237] T dfdb2e11fae24836b81ebb0673610f00 P 2bf9f52803274b3389d7e54375b42ff1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b108b7dddb8d40838456b53ca0eca6d7" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 46677 } } peers { permanent_uuid: "1ff75090686a4259a435c8ba9a041092" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33733 } } peers { permanent_uuid: "2bf9f52803274b3389d7e54375b42ff1" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 42901 } }
I20260811 18:57:04.184127 31171 catalog_manager.cc:5495] T dfdb2e11fae24836b81ebb0673610f00 P 2bf9f52803274b3389d7e54375b42ff1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2bf9f52803274b3389d7e54375b42ff1 (127.29.112.129). New cstate: current_term: 1 leader_uuid: "2bf9f52803274b3389d7e54375b42ff1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b108b7dddb8d40838456b53ca0eca6d7" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 46677 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ff75090686a4259a435c8ba9a041092" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33733 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2bf9f52803274b3389d7e54375b42ff1" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 42901 } health_report { overall_health: HEALTHY } } }
Aug 11 18:57:04 dist-test-slave-h6n0 krb5kdc[31094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474621, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.129@KRBTEST.COM
I20260811 18:57:04.251324 32032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:04.230608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49588 (local address 127.29.112.129:42901)
0811 18:57:04.231016 (+ 408us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:04.231024 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:04.231506 (+ 482us) server_negotiation.cc:407] Connection header received
0811 18:57:04.231633 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:04.231637 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:04.231749 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:04.231912 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:04.234133 (+ 2221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:04.235287 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:04.236912 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:04.237431 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:04.237574 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:04.243710 (+ 6136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:04.243735 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:04.243758 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:04.243819 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:04.247666 (+ 3847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:04.248549 (+ 883us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:04.248560 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:04.248563 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:04.248747 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:04.249499 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:04.249503 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:04.249505 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:04.249947 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:04.250224 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:04.250805 (+ 581us) server_negotiation.cc:299] Negotiation successful
0811 18:57:04.251037 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
W20260811 18:57:04.277002 31517 tablet.cc:2369] T dfdb2e11fae24836b81ebb0673610f00 P 2bf9f52803274b3389d7e54375b42ff1: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 11 18:57:04 dist-test-slave-h6n0 krb5kdc[31094](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474624, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 18:57:04 dist-test-slave-h6n0 krb5kdc[31094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474624, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.29.112.129@KRBTEST.COM
I20260811 18:57:04.321762 32032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:04.302968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49594 (local address 127.29.112.129:42901)
0811 18:57:04.303237 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:04.303247 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:04.303740 (+ 493us) server_negotiation.cc:407] Connection header received
0811 18:57:04.304015 (+ 275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:04.304021 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:04.304247 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:04.304420 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:04.306381 (+ 1961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:04.307288 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:04.309305 (+ 2017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:04.309728 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:04.309841 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:04.314863 (+ 5022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:04.314871 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:04.314884 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:04.314936 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:04.317999 (+ 3063us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:04.319217 (+ 1218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:04.319221 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:04.319224 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:04.319393 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:04.320094 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:04.320100 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:04.320103 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:04.320501 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:04.320805 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:04.321147 (+ 342us) server_negotiation.cc:299] Negotiation successful
0811 18:57:04.321413 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
Aug 11 18:57:04 dist-test-slave-h6n0 krb5kdc[31094](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474624, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 18:57:04 dist-test-slave-h6n0 krb5kdc[31094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474624, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.129@KRBTEST.COM
I20260811 18:57:04.379310 32032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:04.362750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49604 (local address 127.29.112.129:42901)
0811 18:57:04.363019 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:04.363026 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:04.363152 (+ 126us) server_negotiation.cc:407] Connection header received
0811 18:57:04.363306 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:04.363310 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:04.363400 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:04.363502 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:04.364994 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:04.366042 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:04.368057 (+ 2015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:04.368497 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:04.368617 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:04.373894 (+ 5277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:04.373899 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:04.373903 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:04.373943 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:04.376795 (+ 2852us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:04.377440 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:04.377444 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:04.377446 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:04.377578 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:04.377889 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:04.377892 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:04.377894 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:04.378186 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:04.378378 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:04.378760 (+ 382us) server_negotiation.cc:299] Negotiation successful
0811 18:57:04.378919 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119}
W20260811 18:57:04.381146 31365 tablet_service.cc:2908] Rejecting scan request for tablet dfdb2e11fae24836b81ebb0673610f00: Uninitialized: clean time has not yet been initialized
I20260811 18:57:04.384490 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 31295
I20260811 18:57:04.397576 32032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:04.385493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49618 (local address 127.29.112.129:42901)
0811 18:57:04.385659 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:04.385663 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:04.386032 (+ 369us) server_negotiation.cc:407] Connection header received
0811 18:57:04.386138 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:04.386142 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:04.386198 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:04.386272 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:04.387527 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:04.388410 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:04.389532 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:04.389924 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:04.390012 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:04.392210 (+ 2198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:04.392215 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:04.392218 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:04.392248 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:04.395072 (+ 2824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:04.395744 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:04.395748 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:04.395750 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:04.395882 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:04.396308 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:04.396312 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:04.396314 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:04.396562 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:04.396763 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:04.397087 (+ 324us) server_negotiation.cc:299] Negotiation successful
0811 18:57:04.397289 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 18:57:04.398286 31508 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:57:04.491658 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 31295
I20260811 18:57:04.519117 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 31523
Aug 11 18:57:04 dist-test-slave-h6n0 krb5kdc[31094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474624, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.130@KRBTEST.COM
I20260811 18:57:04.535319 32029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:04.519908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36412 (local address 127.29.112.130:33733)
0811 18:57:04.520076 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:04.520081 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:04.520101 (+ 20us) server_negotiation.cc:407] Connection header received
0811 18:57:04.520249 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:04.520254 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:04.520333 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:04.520432 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:04.521699 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:04.522443 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:04.523906 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:04.524341 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:04.524444 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:04.528886 (+ 4442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:04.528901 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:04.528918 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:04.528964 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:04.532247 (+ 3283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:04.532995 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:04.533003 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:04.533005 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:04.533142 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:04.533573 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:04.533577 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:04.533579 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:04.533991 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:04.534223 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:04.534795 (+ 572us) server_negotiation.cc:299] Negotiation successful
0811 18:57:04.535067 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 18:57:04.536115 31758 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:57:04.628667 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 31523
I20260811 18:57:04.656070 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 31776
Aug 11 18:57:04 dist-test-slave-h6n0 krb5kdc[31094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474624, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.131@KRBTEST.COM
I20260811 18:57:04.675647 32026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:04.657202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53818 (local address 127.29.112.131:46677)
0811 18:57:04.657398 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:04.657402 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:04.657639 (+ 237us) server_negotiation.cc:407] Connection header received
0811 18:57:04.657876 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:04.657883 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:04.657982 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:04.658096 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:04.660206 (+ 2110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:04.661412 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:04.663281 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:04.663930 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:04.664086 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:04.669196 (+ 5110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:04.669212 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:04.669229 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:04.669270 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:04.672213 (+ 2943us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:04.673241 (+ 1028us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:04.673248 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:04.673250 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:04.673379 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:04.673932 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:04.673936 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:04.673939 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:04.674312 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:04.674903 (+ 591us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:04.675198 (+ 295us) server_negotiation.cc:299] Negotiation successful
0811 18:57:04.675366 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 18:57:04.676362 31999 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:57:04.771775 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 31776
I20260811 18:57:04.795815 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 31116
I20260811 18:57:04.797493 31230 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:57:04.911178 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 31116
2026-08-11T18:57:04Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4846 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:57:05 dist-test-slave-h6n0 krb5kdc[32090](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:57:05 dist-test-slave-h6n0 krb5kdc[32090](info): set up 2 sockets
Aug 11 18:57:05 dist-test-slave-h6n0 krb5kdc[32090](info): commencing operation
krb5kdc: starting...
W20260811 18:57:06.276695 30146 mini_kdc.cc:121] Time spent starting KDC: real 1.308s user 0.000s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/krb5kdc/test-user.keytab.
Aug 11 18:57:06 dist-test-slave-h6n0 krb5kdc[32090](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474626, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:57:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:57:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.112.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab.
Entry for principal kudu/127.29.112.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab.
I20260811 18:57:06.504719 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.190:46821
--webserver_interface=127.29.112.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab
--principal=kudu/127.29.112.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:42079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.190:46821
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:57:06.854089 32107 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:57:06.854399 32107 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:57:06.854637 32107 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:57:06.862932 32107 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:57:06.863236 32107 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 18:57:06.863435 32107 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:57:06.863583 32107 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:57:06.863651 32107 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:57:06.863716 32107 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:57:06.874972 32107 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:42079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.112.190:46821
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab
--principal=kudu/127.29.112.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.190:46821
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.112.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:57:06.876305 32107 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:57:06.877718 32107 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:57:06.891186 32117 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:06.891350 32118 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:06.892020 32124 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:57:06.892627 32107 server_base.cc:1034] running on GCE node
I20260811 18:57:06.893464 32107 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:57:06.894979 32107 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:57:06.896225 32107 hybrid_clock.cc:648] HybridClock initialized: now 1786474626896168 us; error 28 us; skew 500 ppm
I20260811 18:57:06.896596 32107 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:57:06 dist-test-slave-h6n0 krb5kdc[32090](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474626, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:57:06.900722 32107 init.cc:377] Logged in from keytab as kudu/127.29.112.190@KRBTEST.COM (short username kudu)
I20260811 18:57:06.903040 32107 webserver.cc:460] Webserver started at http://127.29.112.190:34657/ using document root <none> and password file <none>
I20260811 18:57:06.903784 32107 fs_manager.cc:359] Metadata directory not provided
I20260811 18:57:06.903878 32107 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:57:06.904255 32107 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:57:06.907302 32107 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/minicluster-data/master-0/data/instance:
uuid: "679be82e1e23435fb4820f5c2b5cac04"
format_stamp: "Formatted at 2026-08-11 18:57:06 on dist-test-slave-h6n0"
server_key: "716e89fe40c49cf0d6503317038471a5"
server_key_iv: "3507ed56671f6a9a61832bc73c4a3ad4"
server_key_version: "encryptionkey@0"
I20260811 18:57:06.908133 32107 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/minicluster-data/master-0/wal/instance:
uuid: "679be82e1e23435fb4820f5c2b5cac04"
format_stamp: "Formatted at 2026-08-11 18:57:06 on dist-test-slave-h6n0"
server_key: "716e89fe40c49cf0d6503317038471a5"
server_key_iv: "3507ed56671f6a9a61832bc73c4a3ad4"
server_key_version: "encryptionkey@0"
I20260811 18:57:06.913354 32107 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260811 18:57:06.917754 32142 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:06.919344 32107 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 18:57:06.919543 32107 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/minicluster-data/master-0/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/minicluster-data/master-0/wal
uuid: "679be82e1e23435fb4820f5c2b5cac04"
format_stamp: "Formatted at 2026-08-11 18:57:06 on dist-test-slave-h6n0"
server_key: "716e89fe40c49cf0d6503317038471a5"
server_key_iv: "3507ed56671f6a9a61832bc73c4a3ad4"
server_key_version: "encryptionkey@0"
I20260811 18:57:06.919775 32107 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:57:06.944535 32107 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:57:06.949290 32107 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:57:06.949617 32107 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:57:06.974852 32107 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.190:46821
I20260811 18:57:06.974879 32248 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.190:46821 every 8 connection(s)
I20260811 18:57:06.976403 32107 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
I20260811 18:57:06.979995 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 32107
I20260811 18:57:06.980278 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/minicluster-data/master-0/wal/instance
I20260811 18:57:06.980528 32249 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:06.981161 30146 external_mini_cluster.cc:1428] Setting key 5b44a3d46aeeb6dafc7a193d29ae5b8f
I20260811 18:57:06.994769 32249 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 679be82e1e23435fb4820f5c2b5cac04: Bootstrap starting.
Aug 11 18:57:06 dist-test-slave-h6n0 krb5kdc[32090](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474626, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 18:57:06.998463 32249 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 679be82e1e23435fb4820f5c2b5cac04: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:06.999611 32249 log.cc:824] T 00000000000000000000000000000000 P 679be82e1e23435fb4820f5c2b5cac04: Log is configured to *not* fsync() on all Append() calls
I20260811 18:57:07.002384 32249 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 679be82e1e23435fb4820f5c2b5cac04: No bootstrap required, opened a new log
I20260811 18:57:07.006274 32252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:06.983723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38930 (local address 127.29.112.190:46821)
0811 18:57:06.984591 (+ 868us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:06.984603 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:06.984643 (+ 40us) server_negotiation.cc:407] Connection header received
0811 18:57:06.987073 (+ 2430us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:06.987096 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:06.987496 (+ 400us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:06.987821 (+ 325us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:06.989608 (+ 1787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:06.991011 (+ 1403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:06.992739 (+ 1728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:06.993362 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:06.993499 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:06.998380 (+ 4881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:06.998387 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:06.998402 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:06.998444 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:07.002135 (+ 3691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:07.002912 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:07.002919 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:07.002925 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:07.003099 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:07.003512 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:07.003515 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:07.003517 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:07.004047 (+ 530us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:07.004378 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:07.004842 (+ 464us) server_negotiation.cc:299] Negotiation successful
0811 18:57:07.005291 (+ 449us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":665,"thread_start_us":426,"threads_started":1}
I20260811 18:57:07.010612 32249 raft_consensus.cc:348] T 00000000000000000000000000000000 P 679be82e1e23435fb4820f5c2b5cac04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "679be82e1e23435fb4820f5c2b5cac04" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 46821 } }
I20260811 18:57:07.010985 32249 raft_consensus.cc:374] T 00000000000000000000000000000000 P 679be82e1e23435fb4820f5c2b5cac04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:07.011094 32249 raft_consensus.cc:729] T 00000000000000000000000000000000 P 679be82e1e23435fb4820f5c2b5cac04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 679be82e1e23435fb4820f5c2b5cac04, State: Initialized, Role: FOLLOWER
I20260811 18:57:07.011813 32249 consensus_queue.cc:260] T 00000000000000000000000000000000 P 679be82e1e23435fb4820f5c2b5cac04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "679be82e1e23435fb4820f5c2b5cac04" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 46821 } }
I20260811 18:57:07.012188 32249 raft_consensus.cc:388] T 00000000000000000000000000000000 P 679be82e1e23435fb4820f5c2b5cac04 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:57:07.012369 32249 raft_consensus.cc:482] T 00000000000000000000000000000000 P 679be82e1e23435fb4820f5c2b5cac04 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:57:07.012579 32249 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 679be82e1e23435fb4820f5c2b5cac04 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:07.015250 32249 raft_consensus.cc:504] T 00000000000000000000000000000000 P 679be82e1e23435fb4820f5c2b5cac04 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "679be82e1e23435fb4820f5c2b5cac04" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 46821 } }
I20260811 18:57:07.015882 32249 leader_election.cc:302] T 00000000000000000000000000000000 P 679be82e1e23435fb4820f5c2b5cac04 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 679be82e1e23435fb4820f5c2b5cac04; no voters:
I20260811 18:57:07.016553 32249 leader_election.cc:288] T 00000000000000000000000000000000 P 679be82e1e23435fb4820f5c2b5cac04 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:57:07.016726 32254 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 679be82e1e23435fb4820f5c2b5cac04 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:07.017490 32254 raft_consensus.cc:686] T 00000000000000000000000000000000 P 679be82e1e23435fb4820f5c2b5cac04 [term 1 LEADER]: Becoming Leader. State: Replica: 679be82e1e23435fb4820f5c2b5cac04, State: Running, Role: LEADER
I20260811 18:57:07.018034 32254 consensus_queue.cc:237] T 00000000000000000000000000000000 P 679be82e1e23435fb4820f5c2b5cac04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "679be82e1e23435fb4820f5c2b5cac04" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 46821 } }
I20260811 18:57:07.018841 32249 sys_catalog.cc:564] T 00000000000000000000000000000000 P 679be82e1e23435fb4820f5c2b5cac04 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:57:07.020805 32255 sys_catalog.cc:455] T 00000000000000000000000000000000 P 679be82e1e23435fb4820f5c2b5cac04 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "679be82e1e23435fb4820f5c2b5cac04" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "679be82e1e23435fb4820f5c2b5cac04" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 46821 } } }
I20260811 18:57:07.021212 32255 sys_catalog.cc:458] T 00000000000000000000000000000000 P 679be82e1e23435fb4820f5c2b5cac04 [sys.catalog]: This master's current role is: LEADER
I20260811 18:57:07.022740 32256 sys_catalog.cc:455] T 00000000000000000000000000000000 P 679be82e1e23435fb4820f5c2b5cac04 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 679be82e1e23435fb4820f5c2b5cac04. Latest consensus state: current_term: 1 leader_uuid: "679be82e1e23435fb4820f5c2b5cac04" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "679be82e1e23435fb4820f5c2b5cac04" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 46821 } } }
I20260811 18:57:07.023229 32256 sys_catalog.cc:458] T 00000000000000000000000000000000 P 679be82e1e23435fb4820f5c2b5cac04 [sys.catalog]: This master's current role is: LEADER
I20260811 18:57:07.023548 32267 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:57:07.029500 32267 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:57:07.038908 32267 catalog_manager.cc:1347] Generated new cluster ID: 5bd378796e12476e9a3e60af5b8c5873
I20260811 18:57:07.039043 32267 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:57:07.063058 32267 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:57:07.065398 32267 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:57:07.077306 32267 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 679be82e1e23435fb4820f5c2b5cac04: Generated new TSK 0
I20260811 18:57:07.078334 32267 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.112.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.129.keytab.
Entry for principal kudu/127.29.112.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.129.keytab.
I20260811 18:57:07.127741 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.129:0
--local_ip_for_outbound_sockets=127.29.112.129
--webserver_interface=127.29.112.129
--webserver_port=0
--tserver_master_addrs=127.29.112.190:46821
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.129.keytab
--principal=kudu/127.29.112.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:42079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:57:07.476708 32292 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:57:07.476985 32292 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:57:07.477149 32292 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:57:07.484297 32292 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:57:07.484400 32292 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:57:07.484581 32292 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 18:57:07.484690 32292 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.129
I20260811 18:57:07.494362 32292 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:42079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.129.keytab
--principal=kudu/127.29.112.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.112.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.112.190:46821
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.129
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:57:07.495677 32292 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:57:07.497179 32292 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:57:07.510829 32300 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:07.510969 32301 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:07.510969 32304 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:57:07.512001 32292 server_base.cc:1034] running on GCE node
I20260811 18:57:07.512564 32292 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:57:07.513453 32292 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:57:07.514657 32292 hybrid_clock.cc:648] HybridClock initialized: now 1786474627514582 us; error 36 us; skew 500 ppm
I20260811 18:57:07.514954 32292 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:57:07 dist-test-slave-h6n0 krb5kdc[32090](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474627, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:57:07.518252 32292 init.cc:377] Logged in from keytab as kudu/127.29.112.129@KRBTEST.COM (short username kudu)
I20260811 18:57:07.519822 32292 webserver.cc:460] Webserver started at http://127.29.112.129:46135/ using document root <none> and password file <none>
I20260811 18:57:07.520591 32292 fs_manager.cc:359] Metadata directory not provided
I20260811 18:57:07.520680 32292 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:57:07.520936 32292 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:57:07.524022 32292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/minicluster-data/ts-0/data/instance:
uuid: "4610d1c6c23e4fae9be00c51787db5e1"
format_stamp: "Formatted at 2026-08-11 18:57:07 on dist-test-slave-h6n0"
server_key: "06a5e3b34f13612582786146df4be9ad"
server_key_iv: "88a2adfd8e420a2744967391ed24ec8e"
server_key_version: "encryptionkey@0"
I20260811 18:57:07.524829 32292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/minicluster-data/ts-0/wal/instance:
uuid: "4610d1c6c23e4fae9be00c51787db5e1"
format_stamp: "Formatted at 2026-08-11 18:57:07 on dist-test-slave-h6n0"
server_key: "06a5e3b34f13612582786146df4be9ad"
server_key_iv: "88a2adfd8e420a2744967391ed24ec8e"
server_key_version: "encryptionkey@0"
I20260811 18:57:07.529933 32292 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260811 18:57:07.533301 32316 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:07.534739 32292 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:57:07.534910 32292 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/minicluster-data/ts-0/wal
uuid: "4610d1c6c23e4fae9be00c51787db5e1"
format_stamp: "Formatted at 2026-08-11 18:57:07 on dist-test-slave-h6n0"
server_key: "06a5e3b34f13612582786146df4be9ad"
server_key_iv: "88a2adfd8e420a2744967391ed24ec8e"
server_key_version: "encryptionkey@0"
I20260811 18:57:07.535136 32292 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:57:07.555830 32292 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:57:07.560405 32292 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:57:07.560709 32292 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:57:07.563952 32292 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:57:07.564077 32292 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:07.564237 32292 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:57:07.564311 32292 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 18:57:07 dist-test-slave-h6n0 krb5kdc[32090](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474627, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.129@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 18:57:07.591349 32330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:07.564465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.129:55731 (local address 127.29.112.190:46821)
0811 18:57:07.565383 (+ 918us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:07.565388 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:07.566262 (+ 874us) server_negotiation.cc:407] Connection header received
0811 18:57:07.568903 (+ 2641us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:07.568913 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:07.569012 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:07.569357 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:07.572169 (+ 2812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:07.573609 (+ 1440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:07.576858 (+ 3249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:07.577388 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:07.577581 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:07.582674 (+ 5093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:07.582684 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:07.582691 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:07.582743 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:07.586008 (+ 3265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:07.587451 (+ 1443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:07.587459 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:07.587465 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:07.587657 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:07.588374 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:07.588383 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:07.588388 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:07.588827 (+ 439us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:07.589028 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:07.590299 (+ 1271us) server_negotiation.cc:299] Negotiation successful
0811 18:57:07.590771 (+ 472us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":768,"thread_start_us":664,"threads_started":1}
I20260811 18:57:07.592087 32331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:07.565172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:46821 (local address 127.29.112.129:55731)
0811 18:57:07.566075 (+ 903us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:07.566126 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:07.568619 (+ 2493us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:07.569821 (+ 1202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:07.569840 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:07.570881 (+ 1041us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:57:07.571930 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:07.571968 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:07.573946 (+ 1978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:07.573951 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:07.576653 (+ 2702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:07.576673 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:07.577720 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:07.577731 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:07.578103 (+ 372us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:07.579048 (+ 945us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:57:07.579098 (+ 50us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:57:07.582280 (+ 3182us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:57:07.586198 (+ 3918us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:07.586212 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:07.586243 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:07.587242 (+ 999us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:07.587819 (+ 577us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:07.587827 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:07.587833 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:07.588191 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:07.589159 (+ 968us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:57:07.589175 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:57:07.589812 (+ 637us) client_negotiation.cc:769] Sending connection context
0811 18:57:07.590167 (+ 355us) client_negotiation.cc:240] Negotiation successful
0811 18:57:07.590833 (+ 666us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":671,"thread_start_us":433,"threads_started":1}
I20260811 18:57:07.624166 32330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:07.606138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.129:37467 (local address 127.29.112.190:46821)
0811 18:57:07.606382 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:07.606388 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:07.607306 (+ 918us) server_negotiation.cc:407] Connection header received
0811 18:57:07.607544 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:07.607548 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:07.607619 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:07.607759 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:07.609764 (+ 2005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:07.610822 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:07.612168 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:07.612779 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:07.612927 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:07.616701 (+ 3774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:07.616708 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:07.616713 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:07.616754 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:07.620288 (+ 3534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:07.621357 (+ 1069us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:07.621365 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:07.621371 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:07.621560 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:07.622078 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:07.622085 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:07.622088 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:07.622518 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:07.622688 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:07.623659 (+ 971us) server_negotiation.cc:299] Negotiation successful
0811 18:57:07.623910 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260811 18:57:07.624977 32451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:07.606185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:46821 (local address 127.29.112.129:37467)
0811 18:57:07.607135 (+ 950us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:07.607169 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:07.607371 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:07.607804 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:07.607814 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:07.608966 (+ 1152us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:57:07.609523 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:07.609543 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:07.611002 (+ 1459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:07.611006 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:07.612018 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:07.612031 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:07.613219 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:07.613329 (+ 110us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:07.613620 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:07.614659 (+ 1039us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:57:07.614693 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:57:07.616465 (+ 1772us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:57:07.620457 (+ 3992us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:07.620464 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:07.620469 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:07.621201 (+ 732us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:07.621713 (+ 512us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:07.621721 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:07.621724 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:07.621949 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:07.622653 (+ 704us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:57:07.622662 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:57:07.622954 (+ 292us) client_negotiation.cc:769] Sending connection context
0811 18:57:07.623148 (+ 194us) client_negotiation.cc:240] Negotiation successful
0811 18:57:07.624472 (+ 1324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":736,"thread_start_us":515,"threads_started":1}
I20260811 18:57:07.629707 32292 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.129:36717
I20260811 18:57:07.629951 32539 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.129:36717 every 8 connection(s)
I20260811 18:57:07.631074 32292 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/minicluster-data/ts-0/data/info.pb
I20260811 18:57:07.633905 32540 heartbeater.cc:347] Connected to a master server at 127.29.112.190:46821
I20260811 18:57:07.634158 32540 heartbeater.cc:464] Registering TS with master...
I20260811 18:57:07.634963 32540 heartbeater.cc:511] Master 127.29.112.190:46821 requested a full tablet report, sending...
I20260811 18:57:07.636917 32171 ts_manager.cc:194] Registered new tserver with Master: 4610d1c6c23e4fae9be00c51787db5e1 (127.29.112.129:36717)
I20260811 18:57:07.638351 32171 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.129@KRBTEST.COM'} at 127.29.112.129:55731
I20260811 18:57:07.639549 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 32292
I20260811 18:57:07.639794 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/minicluster-data/ts-0/wal/instance
I20260811 18:57:07.640475 30146 external_mini_cluster.cc:1428] Setting key 2c8fc99965394b0fa8524b6cf561c387
WARNING: no policy specified for kudu/127.29.112.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.130.keytab.
Entry for principal kudu/127.29.112.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.130.keytab.
I20260811 18:57:07.696015 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.130:0
--local_ip_for_outbound_sockets=127.29.112.130
--webserver_interface=127.29.112.130
--webserver_port=0
--tserver_master_addrs=127.29.112.190:46821
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.130.keytab
--principal=kudu/127.29.112.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:42079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:57:08.035549 32547 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:57:08.035836 32547 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:57:08.035966 32547 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:57:08.043942 32547 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:57:08.044055 32547 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:57:08.044201 32547 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 18:57:08.044324 32547 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.130
I20260811 18:57:08.054456 32547 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:42079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.130.keytab
--principal=kudu/127.29.112.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.112.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.112.190:46821
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.130
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:57:08.056579 32547 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:57:08.058070 32547 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:57:08.072409 32555 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:08.072570 32558 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:08.072616 32556 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:57:08.073402 32547 server_base.cc:1034] running on GCE node
I20260811 18:57:08.074282 32547 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:57:08.075786 32547 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:57:08.077157 32547 hybrid_clock.cc:648] HybridClock initialized: now 1786474628077071 us; error 42 us; skew 500 ppm
I20260811 18:57:08.077661 32547 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:57:08 dist-test-slave-h6n0 krb5kdc[32090](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474628, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:57:08.085857 32547 init.cc:377] Logged in from keytab as kudu/127.29.112.130@KRBTEST.COM (short username kudu)
I20260811 18:57:08.087662 32547 webserver.cc:460] Webserver started at http://127.29.112.130:45609/ using document root <none> and password file <none>
I20260811 18:57:08.088369 32547 fs_manager.cc:359] Metadata directory not provided
I20260811 18:57:08.088460 32547 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:57:08.088752 32547 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:57:08.091746 32547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/minicluster-data/ts-1/data/instance:
uuid: "0273b8a84b2144519add9ca5bff112c8"
format_stamp: "Formatted at 2026-08-11 18:57:08 on dist-test-slave-h6n0"
server_key: "9312727bb4b9651e5624da2225b14d7d"
server_key_iv: "5eac7d1e3983f759d095cb5e99403534"
server_key_version: "encryptionkey@0"
I20260811 18:57:08.092480 32547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/minicluster-data/ts-1/wal/instance:
uuid: "0273b8a84b2144519add9ca5bff112c8"
format_stamp: "Formatted at 2026-08-11 18:57:08 on dist-test-slave-h6n0"
server_key: "9312727bb4b9651e5624da2225b14d7d"
server_key_iv: "5eac7d1e3983f759d095cb5e99403534"
server_key_version: "encryptionkey@0"
I20260811 18:57:08.098042 32547 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 18:57:08.101438 32574 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:08.102744 32547 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:57:08.102926 32547 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/minicluster-data/ts-1/wal
uuid: "0273b8a84b2144519add9ca5bff112c8"
format_stamp: "Formatted at 2026-08-11 18:57:08 on dist-test-slave-h6n0"
server_key: "9312727bb4b9651e5624da2225b14d7d"
server_key_iv: "5eac7d1e3983f759d095cb5e99403534"
server_key_version: "encryptionkey@0"
I20260811 18:57:08.103134 32547 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:57:08.118060 32547 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:57:08.122350 32547 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:57:08.122649 32547 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:57:08.126091 32547 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:57:08.126268 32547 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:08.126488 32547 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:57:08.126607 32547 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
Aug 11 18:57:08 dist-test-slave-h6n0 krb5kdc[32090](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474628, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.130@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 18:57:08.151351 32586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:08.126586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.130:35405 (local address 127.29.112.190:46821)
0811 18:57:08.127094 (+ 508us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:08.127101 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:08.128458 (+ 1357us) server_negotiation.cc:407] Connection header received
0811 18:57:08.130847 (+ 2389us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:08.130853 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:08.130964 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:08.131106 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:08.133267 (+ 2161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:08.134807 (+ 1540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:08.137667 (+ 2860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:08.138502 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:08.138703 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:08.144495 (+ 5792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:08.144502 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:08.144510 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:08.144565 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:08.148156 (+ 3591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:08.148911 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:08.148917 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:08.148920 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:08.149061 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:08.149525 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:08.149533 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:08.149536 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:08.149817 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:08.149982 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:08.150734 (+ 752us) server_negotiation.cc:299] Negotiation successful
0811 18:57:08.150962 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":252,"threads_started":1}
I20260811 18:57:08.152926 32589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:08.127267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:46821 (local address 127.29.112.130:35405)
0811 18:57:08.128270 (+ 1003us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:08.128323 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:08.130502 (+ 2179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:08.131361 (+ 859us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:08.131370 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:08.131937 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:57:08.132934 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:08.132970 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:08.135317 (+ 2347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:08.135345 (+ 28us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:08.137463 (+ 2118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:08.137479 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:08.138885 (+ 1406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:08.138896 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:08.139242 (+ 346us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:08.140031 (+ 789us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:57:08.140071 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:57:08.144217 (+ 4146us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:57:08.148305 (+ 4088us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:08.148315 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:08.148330 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:08.148784 (+ 454us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:08.149187 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:08.149192 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:08.149196 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:08.149406 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:08.149963 (+ 557us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:57:08.149970 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:57:08.150330 (+ 360us) client_negotiation.cc:769] Sending connection context
0811 18:57:08.150647 (+ 317us) client_negotiation.cc:240] Negotiation successful
0811 18:57:08.151226 (+ 579us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":690,"thread_start_us":427,"threads_started":1}
I20260811 18:57:08.176003 32586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:08.159280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.130:35397 (local address 127.29.112.190:46821)
0811 18:57:08.159556 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:08.159565 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:08.160225 (+ 660us) server_negotiation.cc:407] Connection header received
0811 18:57:08.160384 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:08.160392 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:08.160491 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:08.160613 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:08.161494 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:08.162893 (+ 1399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:08.164927 (+ 2034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:08.165560 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:08.165700 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:08.168124 (+ 2424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:08.168134 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:08.168142 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:08.168185 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:08.172658 (+ 4473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:08.173597 (+ 939us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:08.173604 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:08.173607 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:08.173758 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:08.174150 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:08.174158 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:08.174162 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:08.174773 (+ 611us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:08.174961 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:08.175418 (+ 457us) server_negotiation.cc:299] Negotiation successful
0811 18:57:08.175659 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 18:57:08.176242 32698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:08.159421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:46821 (local address 127.29.112.130:35397)
0811 18:57:08.160116 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:08.160137 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:08.160263 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:08.160646 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:08.160650 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:08.161071 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:57:08.161355 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:08.161363 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:08.163063 (+ 1700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:08.163071 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:08.164781 (+ 1710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:08.164794 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:08.165751 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:08.165758 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:08.165931 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:08.166676 (+ 745us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:57:08.166696 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:57:08.167834 (+ 1138us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:57:08.172821 (+ 4987us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:08.172825 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:08.172827 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:08.173413 (+ 586us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:08.173885 (+ 472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:08.173892 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:08.173900 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:08.174033 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:08.174910 (+ 877us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:57:08.174916 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:57:08.175122 (+ 206us) client_negotiation.cc:769] Sending connection context
0811 18:57:08.175311 (+ 189us) client_negotiation.cc:240] Negotiation successful
0811 18:57:08.175670 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":538,"thread_start_us":395,"threads_started":1}
I20260811 18:57:08.186462 32547 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.130:33685
I20260811 18:57:08.186502 319 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.130:33685 every 8 connection(s)
I20260811 18:57:08.187814 32547 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/minicluster-data/ts-1/data/info.pb
I20260811 18:57:08.190081 325 heartbeater.cc:347] Connected to a master server at 127.29.112.190:46821
I20260811 18:57:08.190349 325 heartbeater.cc:464] Registering TS with master...
I20260811 18:57:08.190335 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 32547
I20260811 18:57:08.190591 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/minicluster-data/ts-1/wal/instance
I20260811 18:57:08.191052 30146 external_mini_cluster.cc:1428] Setting key b93858519e934f347c0ef0080f9b6757
I20260811 18:57:08.191205 325 heartbeater.cc:511] Master 127.29.112.190:46821 requested a full tablet report, sending...
I20260811 18:57:08.192754 32169 ts_manager.cc:194] Registered new tserver with Master: 0273b8a84b2144519add9ca5bff112c8 (127.29.112.130:33685)
I20260811 18:57:08.193908 32169 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.130@KRBTEST.COM'} at 127.29.112.130:35405
WARNING: no policy specified for kudu/127.29.112.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.131.keytab.
Entry for principal kudu/127.29.112.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.131.keytab.
I20260811 18:57:08.247175 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.131:0
--local_ip_for_outbound_sockets=127.29.112.131
--webserver_interface=127.29.112.131
--webserver_port=0
--tserver_master_addrs=127.29.112.190:46821
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.131.keytab
--principal=kudu/127.29.112.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:42079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:57:08.600950 331 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:57:08.601239 331 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:57:08.601430 331 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:57:08.608852 331 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:57:08.609122 331 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:57:08.609331 331 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 18:57:08.609453 331 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.131
I20260811 18:57:08.619362 331 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:42079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.131.keytab
--principal=kudu/127.29.112.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.112.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.112.190:46821
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.131
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:57:08.621031 331 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:57:08.622409 331 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:57:08.639322 340 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:08.639420 343 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:08.639396 338 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:57:08.640281 331 server_base.cc:1034] running on GCE node
I20260811 18:57:08.640946 331 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:57:08.641983 331 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:57:08.642351 32540 heartbeater.cc:503] Master 127.29.112.190:46821 was elected leader, sending a full tablet report...
I20260811 18:57:08.643270 331 hybrid_clock.cc:648] HybridClock initialized: now 1786474628643193 us; error 35 us; skew 500 ppm
I20260811 18:57:08.643651 331 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:57:08 dist-test-slave-h6n0 krb5kdc[32090](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474628, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:57:08.647235 331 init.cc:377] Logged in from keytab as kudu/127.29.112.131@KRBTEST.COM (short username kudu)
I20260811 18:57:08.648736 331 webserver.cc:460] Webserver started at http://127.29.112.131:35737/ using document root <none> and password file <none>
I20260811 18:57:08.649448 331 fs_manager.cc:359] Metadata directory not provided
I20260811 18:57:08.649544 331 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:57:08.649907 331 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:57:08.653455 331 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/minicluster-data/ts-2/data/instance:
uuid: "dbe1e575579f4a6d86cb269f8ba75686"
format_stamp: "Formatted at 2026-08-11 18:57:08 on dist-test-slave-h6n0"
server_key: "f4d15b7e99cdaca10c479818c698fd12"
server_key_iv: "8366d0667fce30ce1f24fc51dd015d80"
server_key_version: "encryptionkey@0"
I20260811 18:57:08.654464 331 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/minicluster-data/ts-2/wal/instance:
uuid: "dbe1e575579f4a6d86cb269f8ba75686"
format_stamp: "Formatted at 2026-08-11 18:57:08 on dist-test-slave-h6n0"
server_key: "f4d15b7e99cdaca10c479818c698fd12"
server_key_iv: "8366d0667fce30ce1f24fc51dd015d80"
server_key_version: "encryptionkey@0"
I20260811 18:57:08.660265 331 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260811 18:57:08.663820 357 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:08.665303 331 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 18:57:08.665501 331 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/minicluster-data/ts-2/wal
uuid: "dbe1e575579f4a6d86cb269f8ba75686"
format_stamp: "Formatted at 2026-08-11 18:57:08 on dist-test-slave-h6n0"
server_key: "f4d15b7e99cdaca10c479818c698fd12"
server_key_iv: "8366d0667fce30ce1f24fc51dd015d80"
server_key_version: "encryptionkey@0"
I20260811 18:57:08.665738 331 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:57:08.684618 331 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:57:08.689469 331 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:57:08.689774 331 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:57:08.693182 331 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:57:08.693300 331 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:08.693459 331 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:57:08.693542 331 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 18:57:08 dist-test-slave-h6n0 krb5kdc[32090](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474628, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.131@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 18:57:08.719106 373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:08.694955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.131:46491 (local address 127.29.112.190:46821)
0811 18:57:08.695457 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:08.695462 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:08.696745 (+ 1283us) server_negotiation.cc:407] Connection header received
0811 18:57:08.699451 (+ 2706us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:08.699460 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:08.699568 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:08.699720 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:08.702027 (+ 2307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:08.703223 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:08.705482 (+ 2259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:08.706225 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:08.706368 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:08.711755 (+ 5387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:08.711760 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:08.711763 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:08.711797 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:08.715440 (+ 3643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:08.716522 (+ 1082us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:08.716527 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:08.716531 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:08.716666 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:08.717141 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:08.717147 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:08.717152 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:08.717529 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:08.717686 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:08.718440 (+ 754us) server_negotiation.cc:299] Negotiation successful
0811 18:57:08.718691 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":397,"thread_start_us":274,"threads_started":1}
I20260811 18:57:08.720213 375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:08.695647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:46821 (local address 127.29.112.131:46491)
0811 18:57:08.696555 (+ 908us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:08.696604 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:08.699035 (+ 2431us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:08.699998 (+ 963us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:08.700007 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:08.700847 (+ 840us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:57:08.701827 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:08.701868 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:08.703424 (+ 1556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:08.703429 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:08.705296 (+ 1867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:08.705309 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:08.706486 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:08.706497 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:08.706922 (+ 425us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:08.707964 (+ 1042us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:57:08.708001 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:57:08.711502 (+ 3501us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:57:08.715612 (+ 4110us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:08.715627 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:08.715646 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:08.716340 (+ 694us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:08.716793 (+ 453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:08.716798 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:08.716802 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:08.717013 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:08.717667 (+ 654us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:57:08.717679 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:57:08.718065 (+ 386us) client_negotiation.cc:769] Sending connection context
0811 18:57:08.718343 (+ 278us) client_negotiation.cc:240] Negotiation successful
0811 18:57:08.718936 (+ 593us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":661,"thread_start_us":392,"threads_started":1}
I20260811 18:57:08.749728 331 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.131:45795
I20260811 18:57:08.749734 578 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.131:45795 every 8 connection(s)
I20260811 18:57:08.751502 331 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/minicluster-data/ts-2/data/info.pb
I20260811 18:57:08.754967 579 heartbeater.cc:347] Connected to a master server at 127.29.112.190:46821
I20260811 18:57:08.755409 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 331
I20260811 18:57:08.755640 579 heartbeater.cc:464] Registering TS with master...
I20260811 18:57:08.755653 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/minicluster-data/ts-2/wal/instance
I20260811 18:57:08.756284 30146 external_mini_cluster.cc:1428] Setting key defb7154b3e7868b266db232ecb2d738
I20260811 18:57:08.757122 579 heartbeater.cc:511] Master 127.29.112.190:46821 requested a full tablet report, sending...
I20260811 18:57:08.759117 32169 ts_manager.cc:194] Registered new tserver with Master: dbe1e575579f4a6d86cb269f8ba75686 (127.29.112.131:45795)
I20260811 18:57:08.759527 544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:08.738366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:46821 (local address 127.29.112.131:53833)
0811 18:57:08.739483 (+ 1117us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:08.739532 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:08.739715 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:08.740203 (+ 488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:08.740210 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:08.740668 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:57:08.740975 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:08.740986 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:08.742892 (+ 1906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:08.742900 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:08.744810 (+ 1910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:08.744840 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:08.746275 (+ 1435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:08.746284 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:08.746566 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:08.747524 (+ 958us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:57:08.747554 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:57:08.748818 (+ 1264us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:57:08.753793 (+ 4975us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:08.753802 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:08.753809 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:08.754732 (+ 923us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:08.757490 (+ 2758us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:08.757497 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:08.757501 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:08.757714 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:08.758381 (+ 667us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:57:08.758388 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:57:08.758638 (+ 250us) client_negotiation.cc:769] Sending connection context
0811 18:57:08.758789 (+ 151us) client_negotiation.cc:240] Negotiation successful
0811 18:57:08.759110 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":929,"thread_start_us":763,"threads_started":1}
I20260811 18:57:08.760226 30146 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 18:57:08.760253 32169 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.131@KRBTEST.COM'} at 127.29.112.131:46491
I20260811 18:57:08.760478 373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:08.738367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.131:53833 (local address 127.29.112.190:46821)
0811 18:57:08.738665 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:08.738675 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:08.739655 (+ 980us) server_negotiation.cc:407] Connection header received
0811 18:57:08.739879 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:08.739887 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:08.740002 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:08.740140 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:08.741141 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:08.742695 (+ 1554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:08.745014 (+ 2319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:08.745807 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:08.745951 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:08.749111 (+ 3160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:08.749122 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:08.749130 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:08.749183 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:08.753581 (+ 4398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:08.756944 (+ 3363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:08.756950 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:08.756954 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:08.757155 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:08.757832 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:08.757837 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:08.757841 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:08.758231 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:08.758440 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:08.759944 (+ 1504us) server_negotiation.cc:299] Negotiation successful
0811 18:57:08.760127 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
Aug 11 18:57:08 dist-test-slave-h6n0 krb5kdc[32090](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474628, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 18:57:08 dist-test-slave-h6n0 krb5kdc[32090](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474628, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 18:57:08.817871 373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:08.802433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38934 (local address 127.29.112.190:46821)
0811 18:57:08.802712 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:08.802718 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:08.802922 (+ 204us) server_negotiation.cc:407] Connection header received
0811 18:57:08.803163 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:08.803168 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:08.803242 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:08.803355 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:08.804779 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:08.805693 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:08.807207 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:08.807674 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:08.807829 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:08.811838 (+ 4009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:08.811846 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:08.811855 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:08.811891 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:08.814979 (+ 3088us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:08.815681 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:08.815690 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:08.815699 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:08.815844 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:08.816284 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:08.816289 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:08.816291 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:08.816603 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:08.816890 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:08.817244 (+ 354us) server_negotiation.cc:299] Negotiation successful
0811 18:57:08.817503 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118}
I20260811 18:57:08.822193 32169 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38934:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:08.829763 32169 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:08.858429 607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:08.843976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54480 (local address 127.29.112.131:45795)
0811 18:57:08.844751 (+ 775us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:08.844758 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:08.845269 (+ 511us) server_negotiation.cc:407] Connection header received
0811 18:57:08.845367 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:08.845374 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:08.846986 (+ 1612us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:08.847239 (+ 253us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:57:08.848472 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:08.851999 (+ 3527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:08.855672 (+ 3673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:08.857213 (+ 1541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:08.857423 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:08.857615 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:08.857877 (+ 262us) server_negotiation.cc:299] Negotiation successful
0811 18:57:08.858053 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":613,"thread_start_us":435,"threads_started":1}
I20260811 18:57:08.858513 601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:08.843318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.131:45795 (local address 127.0.0.1:54480)
0811 18:57:08.845007 (+ 1689us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:08.845038 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:08.845152 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:08.847580 (+ 2428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:08.847587 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:08.847622 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:57:08.848188 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:08.848203 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:08.852221 (+ 4018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:08.852230 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:08.855510 (+ 3280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:08.855529 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:08.857420 (+ 1891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:08.857426 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:08.857627 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:08.857686 (+ 59us) client_negotiation.cc:769] Sending connection context
0811 18:57:08.857837 (+ 151us) client_negotiation.cc:240] Negotiation successful
0811 18:57:08.858053 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1501,"mutex_wait_us":1,"thread_start_us":831,"threads_started":1}
I20260811 18:57:08.859534 605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:08.842738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43336 (local address 127.29.112.130:33685)
0811 18:57:08.843935 (+ 1197us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:08.843947 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:08.844720 (+ 773us) server_negotiation.cc:407] Connection header received
0811 18:57:08.845048 (+ 328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:08.845065 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:08.845351 (+ 286us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:08.845594 (+ 243us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:57:08.847012 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:08.848965 (+ 1953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:08.856560 (+ 7595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:08.857954 (+ 1394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:08.858139 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:08.858335 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:08.859068 (+ 733us) server_negotiation.cc:299] Negotiation successful
0811 18:57:08.859241 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1041,"thread_start_us":929,"threads_started":1}
I20260811 18:57:08.860378 606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:08.842510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.130:33685 (local address 127.0.0.1:43336)
0811 18:57:08.844556 (+ 2046us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:08.844595 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:08.844795 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:08.845911 (+ 1116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:08.845922 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:08.845981 (+ 59us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:57:08.846750 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:08.846769 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:08.849211 (+ 2442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:08.849219 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:08.856294 (+ 7075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:08.856318 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:08.858566 (+ 2248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:08.858573 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:08.858840 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:08.858876 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 18:57:08.859817 (+ 941us) client_negotiation.cc:240] Negotiation successful
0811 18:57:08.859968 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1742,"mutex_wait_us":2,"thread_start_us":1601,"threads_started":1}
I20260811 18:57:08.863552 457 tablet_service.cc:1467] Processing CreateTablet for tablet 615b8ee5474d439588a26268fc774623 (DEFAULT_TABLE table=test-table [id=f425f6f95c9d4da29a828d10594ab89d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:08.863864 32668 tablet_service.cc:1467] Processing CreateTablet for tablet 615b8ee5474d439588a26268fc774623 (DEFAULT_TABLE table=test-table [id=f425f6f95c9d4da29a828d10594ab89d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:08.865083 457 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 615b8ee5474d439588a26268fc774623. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:08.865226 32668 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 615b8ee5474d439588a26268fc774623. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:08.865559 612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:08.845360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35384 (local address 127.29.112.129:36717)
0811 18:57:08.853671 (+ 8311us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:08.853681 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:08.856817 (+ 3136us) server_negotiation.cc:407] Connection header received
0811 18:57:08.856955 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:08.856964 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:08.857233 (+ 269us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:08.857440 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:57:08.858415 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:08.860261 (+ 1846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:08.863055 (+ 2794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:08.864511 (+ 1456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:08.864701 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:08.864900 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:08.865102 (+ 202us) server_negotiation.cc:299] Negotiation successful
0811 18:57:08.865277 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8150,"thread_start_us":8025,"threads_started":1}
I20260811 18:57:08.865849 608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:08.845037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:36717 (local address 127.0.0.1:35384)
0811 18:57:08.856674 (+ 11637us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:08.856695 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:08.856826 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:08.857656 (+ 830us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:08.857661 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:08.857690 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:57:08.858185 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:08.858204 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:08.860407 (+ 2203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:08.860415 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:08.862867 (+ 2452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:08.862883 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:08.864720 (+ 1837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:08.864726 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:08.864903 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:08.864936 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 18:57:08.865043 (+ 107us) client_negotiation.cc:240] Negotiation successful
0811 18:57:08.865269 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11518,"thread_start_us":11322,"threads_started":1}
I20260811 18:57:08.874143 32414 tablet_service.cc:1467] Processing CreateTablet for tablet 615b8ee5474d439588a26268fc774623 (DEFAULT_TABLE table=test-table [id=f425f6f95c9d4da29a828d10594ab89d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:08.875928 32414 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 615b8ee5474d439588a26268fc774623. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:08.879988 613 tablet_bootstrap.cc:492] T 615b8ee5474d439588a26268fc774623 P 0273b8a84b2144519add9ca5bff112c8: Bootstrap starting.
I20260811 18:57:08.883302 615 tablet_bootstrap.cc:492] T 615b8ee5474d439588a26268fc774623 P dbe1e575579f4a6d86cb269f8ba75686: Bootstrap starting.
I20260811 18:57:08.883701 613 tablet_bootstrap.cc:654] T 615b8ee5474d439588a26268fc774623 P 0273b8a84b2144519add9ca5bff112c8: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:08.885097 613 log.cc:824] T 615b8ee5474d439588a26268fc774623 P 0273b8a84b2144519add9ca5bff112c8: Log is configured to *not* fsync() on all Append() calls
I20260811 18:57:08.887121 615 tablet_bootstrap.cc:654] T 615b8ee5474d439588a26268fc774623 P dbe1e575579f4a6d86cb269f8ba75686: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:08.888630 615 log.cc:824] T 615b8ee5474d439588a26268fc774623 P dbe1e575579f4a6d86cb269f8ba75686: Log is configured to *not* fsync() on all Append() calls
I20260811 18:57:08.889657 613 tablet_bootstrap.cc:492] T 615b8ee5474d439588a26268fc774623 P 0273b8a84b2144519add9ca5bff112c8: No bootstrap required, opened a new log
I20260811 18:57:08.890024 613 ts_tablet_manager.cc:1397] T 615b8ee5474d439588a26268fc774623 P 0273b8a84b2144519add9ca5bff112c8: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.004s
I20260811 18:57:08.893062 615 tablet_bootstrap.cc:492] T 615b8ee5474d439588a26268fc774623 P dbe1e575579f4a6d86cb269f8ba75686: No bootstrap required, opened a new log
I20260811 18:57:08.893332 615 ts_tablet_manager.cc:1397] T 615b8ee5474d439588a26268fc774623 P dbe1e575579f4a6d86cb269f8ba75686: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.004s
I20260811 18:57:08.894919 620 tablet_bootstrap.cc:492] T 615b8ee5474d439588a26268fc774623 P 4610d1c6c23e4fae9be00c51787db5e1: Bootstrap starting.
I20260811 18:57:08.898051 620 tablet_bootstrap.cc:654] T 615b8ee5474d439588a26268fc774623 P 4610d1c6c23e4fae9be00c51787db5e1: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:08.899290 620 log.cc:824] T 615b8ee5474d439588a26268fc774623 P 4610d1c6c23e4fae9be00c51787db5e1: Log is configured to *not* fsync() on all Append() calls
I20260811 18:57:08.901043 613 raft_consensus.cc:348] T 615b8ee5474d439588a26268fc774623 P 0273b8a84b2144519add9ca5bff112c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0273b8a84b2144519add9ca5bff112c8" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33685 } } peers { permanent_uuid: "4610d1c6c23e4fae9be00c51787db5e1" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 36717 } } peers { permanent_uuid: "dbe1e575579f4a6d86cb269f8ba75686" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45795 } }
I20260811 18:57:08.901479 613 raft_consensus.cc:374] T 615b8ee5474d439588a26268fc774623 P 0273b8a84b2144519add9ca5bff112c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:08.901584 613 raft_consensus.cc:729] T 615b8ee5474d439588a26268fc774623 P 0273b8a84b2144519add9ca5bff112c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0273b8a84b2144519add9ca5bff112c8, State: Initialized, Role: FOLLOWER
I20260811 18:57:08.902035 620 tablet_bootstrap.cc:492] T 615b8ee5474d439588a26268fc774623 P 4610d1c6c23e4fae9be00c51787db5e1: No bootstrap required, opened a new log
I20260811 18:57:08.902307 620 ts_tablet_manager.cc:1397] T 615b8ee5474d439588a26268fc774623 P 4610d1c6c23e4fae9be00c51787db5e1: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260811 18:57:08.902320 613 consensus_queue.cc:260] T 615b8ee5474d439588a26268fc774623 P 0273b8a84b2144519add9ca5bff112c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0273b8a84b2144519add9ca5bff112c8" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33685 } } peers { permanent_uuid: "4610d1c6c23e4fae9be00c51787db5e1" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 36717 } } peers { permanent_uuid: "dbe1e575579f4a6d86cb269f8ba75686" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45795 } }
I20260811 18:57:08.903466 325 heartbeater.cc:503] Master 127.29.112.190:46821 was elected leader, sending a full tablet report...
I20260811 18:57:08.904147 613 ts_tablet_manager.cc:1428] T 615b8ee5474d439588a26268fc774623 P 0273b8a84b2144519add9ca5bff112c8: Time spent starting tablet: real 0.014s user 0.011s sys 0.003s
I20260811 18:57:08.903949 615 raft_consensus.cc:348] T 615b8ee5474d439588a26268fc774623 P dbe1e575579f4a6d86cb269f8ba75686 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0273b8a84b2144519add9ca5bff112c8" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33685 } } peers { permanent_uuid: "4610d1c6c23e4fae9be00c51787db5e1" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 36717 } } peers { permanent_uuid: "dbe1e575579f4a6d86cb269f8ba75686" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45795 } }
I20260811 18:57:08.904394 615 raft_consensus.cc:374] T 615b8ee5474d439588a26268fc774623 P dbe1e575579f4a6d86cb269f8ba75686 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:08.904501 615 raft_consensus.cc:729] T 615b8ee5474d439588a26268fc774623 P dbe1e575579f4a6d86cb269f8ba75686 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbe1e575579f4a6d86cb269f8ba75686, State: Initialized, Role: FOLLOWER
I20260811 18:57:08.905099 615 consensus_queue.cc:260] T 615b8ee5474d439588a26268fc774623 P dbe1e575579f4a6d86cb269f8ba75686 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0273b8a84b2144519add9ca5bff112c8" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33685 } } peers { permanent_uuid: "4610d1c6c23e4fae9be00c51787db5e1" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 36717 } } peers { permanent_uuid: "dbe1e575579f4a6d86cb269f8ba75686" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45795 } }
I20260811 18:57:08.906234 579 heartbeater.cc:503] Master 127.29.112.190:46821 was elected leader, sending a full tablet report...
I20260811 18:57:08.906876 615 ts_tablet_manager.cc:1428] T 615b8ee5474d439588a26268fc774623 P dbe1e575579f4a6d86cb269f8ba75686: Time spent starting tablet: real 0.013s user 0.015s sys 0.000s
I20260811 18:57:08.913654 620 raft_consensus.cc:348] T 615b8ee5474d439588a26268fc774623 P 4610d1c6c23e4fae9be00c51787db5e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0273b8a84b2144519add9ca5bff112c8" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33685 } } peers { permanent_uuid: "4610d1c6c23e4fae9be00c51787db5e1" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 36717 } } peers { permanent_uuid: "dbe1e575579f4a6d86cb269f8ba75686" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45795 } }
I20260811 18:57:08.914021 620 raft_consensus.cc:374] T 615b8ee5474d439588a26268fc774623 P 4610d1c6c23e4fae9be00c51787db5e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:08.914095 620 raft_consensus.cc:729] T 615b8ee5474d439588a26268fc774623 P 4610d1c6c23e4fae9be00c51787db5e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4610d1c6c23e4fae9be00c51787db5e1, State: Initialized, Role: FOLLOWER
I20260811 18:57:08.914605 620 consensus_queue.cc:260] T 615b8ee5474d439588a26268fc774623 P 4610d1c6c23e4fae9be00c51787db5e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0273b8a84b2144519add9ca5bff112c8" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33685 } } peers { permanent_uuid: "4610d1c6c23e4fae9be00c51787db5e1" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 36717 } } peers { permanent_uuid: "dbe1e575579f4a6d86cb269f8ba75686" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45795 } }
I20260811 18:57:08.916144 620 ts_tablet_manager.cc:1428] T 615b8ee5474d439588a26268fc774623 P 4610d1c6c23e4fae9be00c51787db5e1: Time spent starting tablet: real 0.014s user 0.009s sys 0.004s
W20260811 18:57:08.940109 327 tablet.cc:2369] T 615b8ee5474d439588a26268fc774623 P 0273b8a84b2144519add9ca5bff112c8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 18:57:09.007392 583 tablet.cc:2369] T 615b8ee5474d439588a26268fc774623 P dbe1e575579f4a6d86cb269f8ba75686: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:57:09.081033 623 raft_consensus.cc:482] T 615b8ee5474d439588a26268fc774623 P dbe1e575579f4a6d86cb269f8ba75686 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:09.081457 623 raft_consensus.cc:504] T 615b8ee5474d439588a26268fc774623 P dbe1e575579f4a6d86cb269f8ba75686 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0273b8a84b2144519add9ca5bff112c8" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33685 } } peers { permanent_uuid: "4610d1c6c23e4fae9be00c51787db5e1" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 36717 } } peers { permanent_uuid: "dbe1e575579f4a6d86cb269f8ba75686" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45795 } }
I20260811 18:57:09.083279 623 leader_election.cc:288] T 615b8ee5474d439588a26268fc774623 P dbe1e575579f4a6d86cb269f8ba75686 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0273b8a84b2144519add9ca5bff112c8 (127.29.112.130:33685), 4610d1c6c23e4fae9be00c51787db5e1 (127.29.112.129:36717)
I20260811 18:57:09.089231 375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:09.083837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.130:33685 (local address 127.29.112.131:36389)
0811 18:57:09.084089 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:09.084112 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:09.084243 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:09.084909 (+ 666us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:09.084914 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:09.084944 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:57:09.085169 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:09.085179 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:09.086181 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:09.086188 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:09.087445 (+ 1257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:09.087456 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:09.088562 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:09.088568 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:09.088798 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:09.088821 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 18:57:09.088948 (+ 127us) client_negotiation.cc:240] Negotiation successful
0811 18:57:09.089041 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99}
I20260811 18:57:09.089439 605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:09.083848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.131:36389 (local address 127.29.112.130:33685)
0811 18:57:09.084075 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:09.084082 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:09.084227 (+ 145us) server_negotiation.cc:407] Connection header received
0811 18:57:09.084449 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:09.084457 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:09.084591 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:09.084695 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:57:09.085321 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:09.086040 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:09.087612 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:09.088400 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:09.088573 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:09.088650 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:09.089002 (+ 352us) server_negotiation.cc:299] Negotiation successful
0811 18:57:09.089152 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 18:57:09.090266 32707 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "615b8ee5474d439588a26268fc774623" candidate_uuid: "dbe1e575579f4a6d86cb269f8ba75686" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0273b8a84b2144519add9ca5bff112c8" is_pre_election: true
I20260811 18:57:09.090788 32707 raft_consensus.cc:2454] T 615b8ee5474d439588a26268fc774623 P 0273b8a84b2144519add9ca5bff112c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe1e575579f4a6d86cb269f8ba75686 in term 0.
I20260811 18:57:09.091161 612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:09.084380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.131:56887 (local address 127.29.112.129:36717)
0811 18:57:09.084596 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:09.084602 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:09.085262 (+ 660us) server_negotiation.cc:407] Connection header received
0811 18:57:09.085474 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:09.085484 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:09.085552 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:09.085650 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:57:09.086202 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:09.087055 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:09.088844 (+ 1789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:09.090143 (+ 1299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:09.090334 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:09.090424 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:09.090774 (+ 350us) server_negotiation.cc:299] Negotiation successful
0811 18:57:09.090903 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 18:57:09.091233 627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:09.084218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:36717 (local address 127.29.112.131:56887)
0811 18:57:09.085154 (+ 936us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:09.085171 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:09.085323 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:09.085681 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:09.085686 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:09.085713 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:57:09.086051 (+ 338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:09.086060 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:09.087207 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:09.087213 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:09.088664 (+ 1451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:09.088676 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:09.090373 (+ 1697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:09.090380 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:09.090606 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:09.090628 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 18:57:09.090736 (+ 108us) client_negotiation.cc:240] Negotiation successful
0811 18:57:09.090875 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":804,"thread_start_us":576,"threads_started":1}
I20260811 18:57:09.091482 361 leader_election.cc:302] T 615b8ee5474d439588a26268fc774623 P dbe1e575579f4a6d86cb269f8ba75686 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0273b8a84b2144519add9ca5bff112c8, dbe1e575579f4a6d86cb269f8ba75686; no voters:
I20260811 18:57:09.091917 623 raft_consensus.cc:2781] T 615b8ee5474d439588a26268fc774623 P dbe1e575579f4a6d86cb269f8ba75686 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:09.092096 623 raft_consensus.cc:482] T 615b8ee5474d439588a26268fc774623 P dbe1e575579f4a6d86cb269f8ba75686 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:09.092228 623 raft_consensus.cc:3037] T 615b8ee5474d439588a26268fc774623 P dbe1e575579f4a6d86cb269f8ba75686 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:09.092625 32453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "615b8ee5474d439588a26268fc774623" candidate_uuid: "dbe1e575579f4a6d86cb269f8ba75686" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4610d1c6c23e4fae9be00c51787db5e1" is_pre_election: true
I20260811 18:57:09.093060 32453 raft_consensus.cc:2454] T 615b8ee5474d439588a26268fc774623 P 4610d1c6c23e4fae9be00c51787db5e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe1e575579f4a6d86cb269f8ba75686 in term 0.
I20260811 18:57:09.095114 623 raft_consensus.cc:504] T 615b8ee5474d439588a26268fc774623 P dbe1e575579f4a6d86cb269f8ba75686 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0273b8a84b2144519add9ca5bff112c8" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33685 } } peers { permanent_uuid: "4610d1c6c23e4fae9be00c51787db5e1" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 36717 } } peers { permanent_uuid: "dbe1e575579f4a6d86cb269f8ba75686" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45795 } }
I20260811 18:57:09.095731 623 leader_election.cc:288] T 615b8ee5474d439588a26268fc774623 P dbe1e575579f4a6d86cb269f8ba75686 [CANDIDATE]: Term 1 election: Requested vote from peers 0273b8a84b2144519add9ca5bff112c8 (127.29.112.130:33685), 4610d1c6c23e4fae9be00c51787db5e1 (127.29.112.129:36717)
I20260811 18:57:09.096195 32707 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "615b8ee5474d439588a26268fc774623" candidate_uuid: "dbe1e575579f4a6d86cb269f8ba75686" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0273b8a84b2144519add9ca5bff112c8"
I20260811 18:57:09.096256 32453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "615b8ee5474d439588a26268fc774623" candidate_uuid: "dbe1e575579f4a6d86cb269f8ba75686" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4610d1c6c23e4fae9be00c51787db5e1"
I20260811 18:57:09.096412 32707 raft_consensus.cc:3037] T 615b8ee5474d439588a26268fc774623 P 0273b8a84b2144519add9ca5bff112c8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:09.096438 32453 raft_consensus.cc:3037] T 615b8ee5474d439588a26268fc774623 P 4610d1c6c23e4fae9be00c51787db5e1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:09.099689 32707 raft_consensus.cc:2454] T 615b8ee5474d439588a26268fc774623 P 0273b8a84b2144519add9ca5bff112c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe1e575579f4a6d86cb269f8ba75686 in term 1.
I20260811 18:57:09.099689 32453 raft_consensus.cc:2454] T 615b8ee5474d439588a26268fc774623 P 4610d1c6c23e4fae9be00c51787db5e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe1e575579f4a6d86cb269f8ba75686 in term 1.
I20260811 18:57:09.100240 361 leader_election.cc:302] T 615b8ee5474d439588a26268fc774623 P dbe1e575579f4a6d86cb269f8ba75686 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4610d1c6c23e4fae9be00c51787db5e1, dbe1e575579f4a6d86cb269f8ba75686; no voters:
I20260811 18:57:09.100526 623 raft_consensus.cc:2781] T 615b8ee5474d439588a26268fc774623 P dbe1e575579f4a6d86cb269f8ba75686 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:09.100984 623 raft_consensus.cc:686] T 615b8ee5474d439588a26268fc774623 P dbe1e575579f4a6d86cb269f8ba75686 [term 1 LEADER]: Becoming Leader. State: Replica: dbe1e575579f4a6d86cb269f8ba75686, State: Running, Role: LEADER
I20260811 18:57:09.101445 623 consensus_queue.cc:237] T 615b8ee5474d439588a26268fc774623 P dbe1e575579f4a6d86cb269f8ba75686 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0273b8a84b2144519add9ca5bff112c8" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33685 } } peers { permanent_uuid: "4610d1c6c23e4fae9be00c51787db5e1" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 36717 } } peers { permanent_uuid: "dbe1e575579f4a6d86cb269f8ba75686" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45795 } }
I20260811 18:57:09.106422 32167 catalog_manager.cc:5495] T 615b8ee5474d439588a26268fc774623 P dbe1e575579f4a6d86cb269f8ba75686 reported cstate change: term changed from 0 to 1, leader changed from <none> to dbe1e575579f4a6d86cb269f8ba75686 (127.29.112.131). New cstate: current_term: 1 leader_uuid: "dbe1e575579f4a6d86cb269f8ba75686" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0273b8a84b2144519add9ca5bff112c8" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4610d1c6c23e4fae9be00c51787db5e1" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 36717 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbe1e575579f4a6d86cb269f8ba75686" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45795 } health_report { overall_health: HEALTHY } } }
W20260811 18:57:09.134362 32541 tablet.cc:2369] T 615b8ee5474d439588a26268fc774623 P 4610d1c6c23e4fae9be00c51787db5e1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:57:09.160661 607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:09.154131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43482 (local address 127.29.112.131:45795)
0811 18:57:09.154315 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:09.154320 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:09.154414 (+ 94us) server_negotiation.cc:407] Connection header received
0811 18:57:09.154781 (+ 367us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:09.154784 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:09.154872 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:09.154997 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:57:09.155412 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:09.156272 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:09.157794 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:09.158432 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:09.158614 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:09.159641 (+ 1027us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:57:09.160115 (+ 474us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:57:09.160218 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:09.160377 (+ 159us) server_negotiation.cc:299] Negotiation successful
0811 18:57:09.160446 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 18:57:09.168571 32707 raft_consensus.cc:1261] T 615b8ee5474d439588a26268fc774623 P 0273b8a84b2144519add9ca5bff112c8 [term 1 FOLLOWER]: Refusing update from remote peer dbe1e575579f4a6d86cb269f8ba75686: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:57:09.168658 32453 raft_consensus.cc:1261] T 615b8ee5474d439588a26268fc774623 P 4610d1c6c23e4fae9be00c51787db5e1 [term 1 FOLLOWER]: Refusing update from remote peer dbe1e575579f4a6d86cb269f8ba75686: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:57:09.169596 628 consensus_queue.cc:1035] T 615b8ee5474d439588a26268fc774623 P dbe1e575579f4a6d86cb269f8ba75686 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0273b8a84b2144519add9ca5bff112c8" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:09.169973 623 consensus_queue.cc:1035] T 615b8ee5474d439588a26268fc774623 P dbe1e575579f4a6d86cb269f8ba75686 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4610d1c6c23e4fae9be00c51787db5e1" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 36717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:09.181813 636 mvcc.cc:205] Tried to move back new op lower bound from 7317400081064062976 to 7317400080802959360. Current Snapshot: MvccSnapshot[applied={T|T < 7317400081064062976}]
I20260811 18:57:09.205997 32167 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38934:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:57:09.206314 32167 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38934:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:57:09.210338 32167 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 679be82e1e23435fb4820f5c2b5cac04: Sending DeleteTablet for 3 replicas of tablet 615b8ee5474d439588a26268fc774623
I20260811 18:57:09.211458 32668 tablet_service.cc:1514] Processing DeleteTablet for tablet 615b8ee5474d439588a26268fc774623 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:09 UTC) from {username='kudu', principal='kudu/127.29.112.190@KRBTEST.COM'} at 127.0.0.1:43336
I20260811 18:57:09.211506 457 tablet_service.cc:1514] Processing DeleteTablet for tablet 615b8ee5474d439588a26268fc774623 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:09 UTC) from {username='kudu', principal='kudu/127.29.112.190@KRBTEST.COM'} at 127.0.0.1:54480
I20260811 18:57:09.211539 32414 tablet_service.cc:1514] Processing DeleteTablet for tablet 615b8ee5474d439588a26268fc774623 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:09 UTC) from {username='kudu', principal='kudu/127.29.112.190@KRBTEST.COM'} at 127.0.0.1:35384
I20260811 18:57:09.213363 32167 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38934:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:09.214136 32167 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:09.223832 32412 tablet_service.cc:1467] Processing CreateTablet for tablet 47a79343478f4d3a8c36d9f0e8a87e8d (DEFAULT_TABLE table=test-table [id=44364c171fd2480dbc8637972ba88bb2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:09.224139 455 tablet_service.cc:1467] Processing CreateTablet for tablet 47a79343478f4d3a8c36d9f0e8a87e8d (DEFAULT_TABLE table=test-table [id=44364c171fd2480dbc8637972ba88bb2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:09.224555 32412 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47a79343478f4d3a8c36d9f0e8a87e8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:09.224798 455 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47a79343478f4d3a8c36d9f0e8a87e8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:09.226231 655 tablet_replica.cc:330] T 615b8ee5474d439588a26268fc774623 P dbe1e575579f4a6d86cb269f8ba75686: stopping tablet replica
I20260811 18:57:09.226711 655 raft_consensus.cc:2229] T 615b8ee5474d439588a26268fc774623 P dbe1e575579f4a6d86cb269f8ba75686 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:09.227005 32667 tablet_service.cc:1467] Processing CreateTablet for tablet 47a79343478f4d3a8c36d9f0e8a87e8d (DEFAULT_TABLE table=test-table [id=44364c171fd2480dbc8637972ba88bb2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:09.227970 32667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47a79343478f4d3a8c36d9f0e8a87e8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:09.231523 658 tablet_replica.cc:330] T 615b8ee5474d439588a26268fc774623 P 0273b8a84b2144519add9ca5bff112c8: stopping tablet replica
I20260811 18:57:09.231586 620 tablet_bootstrap.cc:492] T 47a79343478f4d3a8c36d9f0e8a87e8d P 4610d1c6c23e4fae9be00c51787db5e1: Bootstrap starting.
I20260811 18:57:09.232455 658 raft_consensus.cc:2229] T 615b8ee5474d439588a26268fc774623 P 0273b8a84b2144519add9ca5bff112c8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:09.233022 658 raft_consensus.cc:2258] T 615b8ee5474d439588a26268fc774623 P 0273b8a84b2144519add9ca5bff112c8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:09.233433 620 tablet_bootstrap.cc:654] T 47a79343478f4d3a8c36d9f0e8a87e8d P 4610d1c6c23e4fae9be00c51787db5e1: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:09.234546 655 raft_consensus.cc:2258] T 615b8ee5474d439588a26268fc774623 P dbe1e575579f4a6d86cb269f8ba75686 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:09.235800 659 tablet_replica.cc:330] T 615b8ee5474d439588a26268fc774623 P 4610d1c6c23e4fae9be00c51787db5e1: stopping tablet replica
I20260811 18:57:09.235888 613 tablet_bootstrap.cc:492] T 47a79343478f4d3a8c36d9f0e8a87e8d P 0273b8a84b2144519add9ca5bff112c8: Bootstrap starting.
I20260811 18:57:09.236341 659 raft_consensus.cc:2229] T 615b8ee5474d439588a26268fc774623 P 4610d1c6c23e4fae9be00c51787db5e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:09.237901 659 raft_consensus.cc:2258] T 615b8ee5474d439588a26268fc774623 P 4610d1c6c23e4fae9be00c51787db5e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:09.238771 615 tablet_bootstrap.cc:492] T 47a79343478f4d3a8c36d9f0e8a87e8d P dbe1e575579f4a6d86cb269f8ba75686: Bootstrap starting.
I20260811 18:57:09.238837 613 tablet_bootstrap.cc:654] T 47a79343478f4d3a8c36d9f0e8a87e8d P 0273b8a84b2144519add9ca5bff112c8: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:09.240600 615 tablet_bootstrap.cc:654] T 47a79343478f4d3a8c36d9f0e8a87e8d P dbe1e575579f4a6d86cb269f8ba75686: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:09.244212 659 ts_tablet_manager.cc:1905] T 615b8ee5474d439588a26268fc774623 P 4610d1c6c23e4fae9be00c51787db5e1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:09.244657 615 tablet_bootstrap.cc:492] T 47a79343478f4d3a8c36d9f0e8a87e8d P dbe1e575579f4a6d86cb269f8ba75686: No bootstrap required, opened a new log
I20260811 18:57:09.244853 615 ts_tablet_manager.cc:1397] T 47a79343478f4d3a8c36d9f0e8a87e8d P dbe1e575579f4a6d86cb269f8ba75686: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 18:57:09.245713 615 raft_consensus.cc:348] T 47a79343478f4d3a8c36d9f0e8a87e8d P dbe1e575579f4a6d86cb269f8ba75686 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4610d1c6c23e4fae9be00c51787db5e1" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 36717 } } peers { permanent_uuid: "0273b8a84b2144519add9ca5bff112c8" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33685 } } peers { permanent_uuid: "dbe1e575579f4a6d86cb269f8ba75686" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45795 } }
I20260811 18:57:09.246018 615 raft_consensus.cc:374] T 47a79343478f4d3a8c36d9f0e8a87e8d P dbe1e575579f4a6d86cb269f8ba75686 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:09.246173 615 raft_consensus.cc:729] T 47a79343478f4d3a8c36d9f0e8a87e8d P dbe1e575579f4a6d86cb269f8ba75686 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbe1e575579f4a6d86cb269f8ba75686, State: Initialized, Role: FOLLOWER
I20260811 18:57:09.247563 615 consensus_queue.cc:260] T 47a79343478f4d3a8c36d9f0e8a87e8d P dbe1e575579f4a6d86cb269f8ba75686 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4610d1c6c23e4fae9be00c51787db5e1" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 36717 } } peers { permanent_uuid: "0273b8a84b2144519add9ca5bff112c8" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33685 } } peers { permanent_uuid: "dbe1e575579f4a6d86cb269f8ba75686" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45795 } }
I20260811 18:57:09.247779 613 tablet_bootstrap.cc:492] T 47a79343478f4d3a8c36d9f0e8a87e8d P 0273b8a84b2144519add9ca5bff112c8: No bootstrap required, opened a new log
I20260811 18:57:09.248080 613 ts_tablet_manager.cc:1397] T 47a79343478f4d3a8c36d9f0e8a87e8d P 0273b8a84b2144519add9ca5bff112c8: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.005s
I20260811 18:57:09.248337 615 ts_tablet_manager.cc:1428] T 47a79343478f4d3a8c36d9f0e8a87e8d P dbe1e575579f4a6d86cb269f8ba75686: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260811 18:57:09.248579 658 ts_tablet_manager.cc:1905] T 615b8ee5474d439588a26268fc774623 P 0273b8a84b2144519add9ca5bff112c8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:09.248984 613 raft_consensus.cc:348] T 47a79343478f4d3a8c36d9f0e8a87e8d P 0273b8a84b2144519add9ca5bff112c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4610d1c6c23e4fae9be00c51787db5e1" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 36717 } } peers { permanent_uuid: "0273b8a84b2144519add9ca5bff112c8" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33685 } } peers { permanent_uuid: "dbe1e575579f4a6d86cb269f8ba75686" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45795 } }
I20260811 18:57:09.249259 613 raft_consensus.cc:374] T 47a79343478f4d3a8c36d9f0e8a87e8d P 0273b8a84b2144519add9ca5bff112c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:09.249362 613 raft_consensus.cc:729] T 47a79343478f4d3a8c36d9f0e8a87e8d P 0273b8a84b2144519add9ca5bff112c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0273b8a84b2144519add9ca5bff112c8, State: Initialized, Role: FOLLOWER
I20260811 18:57:09.249652 613 consensus_queue.cc:260] T 47a79343478f4d3a8c36d9f0e8a87e8d P 0273b8a84b2144519add9ca5bff112c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4610d1c6c23e4fae9be00c51787db5e1" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 36717 } } peers { permanent_uuid: "0273b8a84b2144519add9ca5bff112c8" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33685 } } peers { permanent_uuid: "dbe1e575579f4a6d86cb269f8ba75686" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45795 } }
I20260811 18:57:09.250869 613 ts_tablet_manager.cc:1428] T 47a79343478f4d3a8c36d9f0e8a87e8d P 0273b8a84b2144519add9ca5bff112c8: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260811 18:57:09.253834 655 ts_tablet_manager.cc:1905] T 615b8ee5474d439588a26268fc774623 P dbe1e575579f4a6d86cb269f8ba75686: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:09.257050 620 tablet_bootstrap.cc:492] T 47a79343478f4d3a8c36d9f0e8a87e8d P 4610d1c6c23e4fae9be00c51787db5e1: No bootstrap required, opened a new log
I20260811 18:57:09.257282 620 ts_tablet_manager.cc:1397] T 47a79343478f4d3a8c36d9f0e8a87e8d P 4610d1c6c23e4fae9be00c51787db5e1: Time spent bootstrapping tablet: real 0.026s user 0.013s sys 0.004s
I20260811 18:57:09.258211 620 raft_consensus.cc:348] T 47a79343478f4d3a8c36d9f0e8a87e8d P 4610d1c6c23e4fae9be00c51787db5e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4610d1c6c23e4fae9be00c51787db5e1" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 36717 } } peers { permanent_uuid: "0273b8a84b2144519add9ca5bff112c8" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33685 } } peers { permanent_uuid: "dbe1e575579f4a6d86cb269f8ba75686" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45795 } }
I20260811 18:57:09.258838 620 raft_consensus.cc:374] T 47a79343478f4d3a8c36d9f0e8a87e8d P 4610d1c6c23e4fae9be00c51787db5e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:09.259272 620 raft_consensus.cc:729] T 47a79343478f4d3a8c36d9f0e8a87e8d P 4610d1c6c23e4fae9be00c51787db5e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4610d1c6c23e4fae9be00c51787db5e1, State: Initialized, Role: FOLLOWER
I20260811 18:57:09.259680 620 consensus_queue.cc:260] T 47a79343478f4d3a8c36d9f0e8a87e8d P 4610d1c6c23e4fae9be00c51787db5e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4610d1c6c23e4fae9be00c51787db5e1" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 36717 } } peers { permanent_uuid: "0273b8a84b2144519add9ca5bff112c8" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33685 } } peers { permanent_uuid: "dbe1e575579f4a6d86cb269f8ba75686" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45795 } }
I20260811 18:57:09.260381 620 ts_tablet_manager.cc:1428] T 47a79343478f4d3a8c36d9f0e8a87e8d P 4610d1c6c23e4fae9be00c51787db5e1: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260811 18:57:09.262913 655 ts_tablet_manager.cc:1918] T 615b8ee5474d439588a26268fc774623 P dbe1e575579f4a6d86cb269f8ba75686: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:57:09.263118 655 log.cc:1196] T 615b8ee5474d439588a26268fc774623 P dbe1e575579f4a6d86cb269f8ba75686: Deleting WAL directory at /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/minicluster-data/ts-2/wal/wals/615b8ee5474d439588a26268fc774623
I20260811 18:57:09.263729 655 ts_tablet_manager.cc:1939] T 615b8ee5474d439588a26268fc774623 P dbe1e575579f4a6d86cb269f8ba75686: Deleting consensus metadata
I20260811 18:57:09.264878 658 ts_tablet_manager.cc:1918] T 615b8ee5474d439588a26268fc774623 P 0273b8a84b2144519add9ca5bff112c8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:57:09.265055 658 log.cc:1196] T 615b8ee5474d439588a26268fc774623 P 0273b8a84b2144519add9ca5bff112c8: Deleting WAL directory at /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/minicluster-data/ts-1/wal/wals/615b8ee5474d439588a26268fc774623
I20260811 18:57:09.265606 658 ts_tablet_manager.cc:1939] T 615b8ee5474d439588a26268fc774623 P 0273b8a84b2144519add9ca5bff112c8: Deleting consensus metadata
I20260811 18:57:09.265668 32146 catalog_manager.cc:4841] TS dbe1e575579f4a6d86cb269f8ba75686 (127.29.112.131:45795): tablet 615b8ee5474d439588a26268fc774623 (table test-table [id=f425f6f95c9d4da29a828d10594ab89d]) successfully deleted
I20260811 18:57:09.267292 32144 catalog_manager.cc:4841] TS 0273b8a84b2144519add9ca5bff112c8 (127.29.112.130:33685): tablet 615b8ee5474d439588a26268fc774623 (table test-table [id=f425f6f95c9d4da29a828d10594ab89d]) successfully deleted
I20260811 18:57:09.268749 659 ts_tablet_manager.cc:1918] T 615b8ee5474d439588a26268fc774623 P 4610d1c6c23e4fae9be00c51787db5e1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:57:09.268919 659 log.cc:1196] T 615b8ee5474d439588a26268fc774623 P 4610d1c6c23e4fae9be00c51787db5e1: Deleting WAL directory at /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/minicluster-data/ts-0/wal/wals/615b8ee5474d439588a26268fc774623
I20260811 18:57:09.269416 659 ts_tablet_manager.cc:1939] T 615b8ee5474d439588a26268fc774623 P 4610d1c6c23e4fae9be00c51787db5e1: Deleting consensus metadata
I20260811 18:57:09.271214 32144 catalog_manager.cc:4841] TS 4610d1c6c23e4fae9be00c51787db5e1 (127.29.112.129:36717): tablet 615b8ee5474d439588a26268fc774623 (table test-table [id=f425f6f95c9d4da29a828d10594ab89d]) successfully deleted
I20260811 18:57:09.426512 624 raft_consensus.cc:482] T 47a79343478f4d3a8c36d9f0e8a87e8d P 4610d1c6c23e4fae9be00c51787db5e1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:09.427029 624 raft_consensus.cc:504] T 47a79343478f4d3a8c36d9f0e8a87e8d P 4610d1c6c23e4fae9be00c51787db5e1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4610d1c6c23e4fae9be00c51787db5e1" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 36717 } } peers { permanent_uuid: "0273b8a84b2144519add9ca5bff112c8" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33685 } } peers { permanent_uuid: "dbe1e575579f4a6d86cb269f8ba75686" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45795 } }
I20260811 18:57:09.429194 624 leader_election.cc:288] T 47a79343478f4d3a8c36d9f0e8a87e8d P 4610d1c6c23e4fae9be00c51787db5e1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0273b8a84b2144519add9ca5bff112c8 (127.29.112.130:33685), dbe1e575579f4a6d86cb269f8ba75686 (127.29.112.131:45795)
I20260811 18:57:09.438016 607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:09.429746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.129:35217 (local address 127.29.112.131:45795)
0811 18:57:09.429977 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:09.429987 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:09.431904 (+ 1917us) server_negotiation.cc:407] Connection header received
0811 18:57:09.432110 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:09.432116 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:09.432201 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:09.432274 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:57:09.433006 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:09.433773 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:09.435834 (+ 2061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:09.436911 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:09.437077 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:09.437153 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:09.437535 (+ 382us) server_negotiation.cc:299] Negotiation successful
0811 18:57:09.437670 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 18:57:09.438105 668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:09.430334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.131:45795 (local address 127.29.112.129:35217)
0811 18:57:09.431765 (+ 1431us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:09.431786 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:09.431920 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:09.432511 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:09.432516 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:09.432541 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:57:09.432856 (+ 315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:09.432864 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:09.433920 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:09.433925 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:09.435665 (+ 1740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:09.435677 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:09.437148 (+ 1471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:09.437154 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:09.437322 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:09.437369 (+ 47us) client_negotiation.cc:769] Sending connection context
0811 18:57:09.437474 (+ 105us) client_negotiation.cc:240] Negotiation successful
0811 18:57:09.437608 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1271,"mutex_wait_us":101,"thread_start_us":1070,"threads_started":1}
I20260811 18:57:09.439126 605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:09.429708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.129:47789 (local address 127.29.112.130:33685)
0811 18:57:09.429900 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:09.429907 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:09.431442 (+ 1535us) server_negotiation.cc:407] Connection header received
0811 18:57:09.431791 (+ 349us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:09.431800 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:09.431895 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:09.431994 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:57:09.432612 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:09.433659 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:09.436301 (+ 2642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:09.437426 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:09.437609 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:09.437674 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:09.438693 (+ 1019us) server_negotiation.cc:299] Negotiation successful
0811 18:57:09.438841 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 18:57:09.439066 492 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47a79343478f4d3a8c36d9f0e8a87e8d" candidate_uuid: "4610d1c6c23e4fae9be00c51787db5e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbe1e575579f4a6d86cb269f8ba75686" is_pre_election: true
I20260811 18:57:09.439602 492 raft_consensus.cc:2454] T 47a79343478f4d3a8c36d9f0e8a87e8d P dbe1e575579f4a6d86cb269f8ba75686 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4610d1c6c23e4fae9be00c51787db5e1 in term 0.
I20260811 18:57:09.439908 667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:09.429672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.130:33685 (local address 127.29.112.129:47789)
0811 18:57:09.431295 (+ 1623us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:09.431322 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:09.431578 (+ 256us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:09.432008 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:09.432013 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:09.432047 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:57:09.432411 (+ 364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:09.432422 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:09.433853 (+ 1431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:09.433858 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:09.436133 (+ 2275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:09.436149 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:09.437838 (+ 1689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:09.437845 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:09.438048 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:09.438503 (+ 455us) client_negotiation.cc:769] Sending connection context
0811 18:57:09.439379 (+ 876us) client_negotiation.cc:240] Negotiation successful
0811 18:57:09.439509 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1460,"mutex_wait_us":1,"thread_start_us":1146,"threads_started":1}
I20260811 18:57:09.440261 32319 leader_election.cc:302] T 47a79343478f4d3a8c36d9f0e8a87e8d P 4610d1c6c23e4fae9be00c51787db5e1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4610d1c6c23e4fae9be00c51787db5e1, dbe1e575579f4a6d86cb269f8ba75686; no voters:
I20260811 18:57:09.440491 32707 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47a79343478f4d3a8c36d9f0e8a87e8d" candidate_uuid: "4610d1c6c23e4fae9be00c51787db5e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0273b8a84b2144519add9ca5bff112c8" is_pre_election: true
I20260811 18:57:09.440696 624 raft_consensus.cc:2781] T 47a79343478f4d3a8c36d9f0e8a87e8d P 4610d1c6c23e4fae9be00c51787db5e1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:09.440734 32707 raft_consensus.cc:2454] T 47a79343478f4d3a8c36d9f0e8a87e8d P 0273b8a84b2144519add9ca5bff112c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4610d1c6c23e4fae9be00c51787db5e1 in term 0.
I20260811 18:57:09.440817 624 raft_consensus.cc:482] T 47a79343478f4d3a8c36d9f0e8a87e8d P 4610d1c6c23e4fae9be00c51787db5e1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:09.440903 624 raft_consensus.cc:3037] T 47a79343478f4d3a8c36d9f0e8a87e8d P 4610d1c6c23e4fae9be00c51787db5e1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:09.444710 624 raft_consensus.cc:504] T 47a79343478f4d3a8c36d9f0e8a87e8d P 4610d1c6c23e4fae9be00c51787db5e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4610d1c6c23e4fae9be00c51787db5e1" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 36717 } } peers { permanent_uuid: "0273b8a84b2144519add9ca5bff112c8" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33685 } } peers { permanent_uuid: "dbe1e575579f4a6d86cb269f8ba75686" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45795 } }
I20260811 18:57:09.445513 624 leader_election.cc:288] T 47a79343478f4d3a8c36d9f0e8a87e8d P 4610d1c6c23e4fae9be00c51787db5e1 [CANDIDATE]: Term 1 election: Requested vote from peers 0273b8a84b2144519add9ca5bff112c8 (127.29.112.130:33685), dbe1e575579f4a6d86cb269f8ba75686 (127.29.112.131:45795)
I20260811 18:57:09.446054 492 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47a79343478f4d3a8c36d9f0e8a87e8d" candidate_uuid: "4610d1c6c23e4fae9be00c51787db5e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbe1e575579f4a6d86cb269f8ba75686"
I20260811 18:57:09.446039 32707 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47a79343478f4d3a8c36d9f0e8a87e8d" candidate_uuid: "4610d1c6c23e4fae9be00c51787db5e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0273b8a84b2144519add9ca5bff112c8"
I20260811 18:57:09.446265 32707 raft_consensus.cc:3037] T 47a79343478f4d3a8c36d9f0e8a87e8d P 0273b8a84b2144519add9ca5bff112c8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:09.446273 492 raft_consensus.cc:3037] T 47a79343478f4d3a8c36d9f0e8a87e8d P dbe1e575579f4a6d86cb269f8ba75686 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:09.449115 492 raft_consensus.cc:2454] T 47a79343478f4d3a8c36d9f0e8a87e8d P dbe1e575579f4a6d86cb269f8ba75686 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4610d1c6c23e4fae9be00c51787db5e1 in term 1.
I20260811 18:57:09.449635 32707 raft_consensus.cc:2454] T 47a79343478f4d3a8c36d9f0e8a87e8d P 0273b8a84b2144519add9ca5bff112c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4610d1c6c23e4fae9be00c51787db5e1 in term 1.
I20260811 18:57:09.449663 32319 leader_election.cc:302] T 47a79343478f4d3a8c36d9f0e8a87e8d P 4610d1c6c23e4fae9be00c51787db5e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4610d1c6c23e4fae9be00c51787db5e1, dbe1e575579f4a6d86cb269f8ba75686; no voters:
I20260811 18:57:09.450014 624 raft_consensus.cc:2781] T 47a79343478f4d3a8c36d9f0e8a87e8d P 4610d1c6c23e4fae9be00c51787db5e1 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:09.450722 624 raft_consensus.cc:686] T 47a79343478f4d3a8c36d9f0e8a87e8d P 4610d1c6c23e4fae9be00c51787db5e1 [term 1 LEADER]: Becoming Leader. State: Replica: 4610d1c6c23e4fae9be00c51787db5e1, State: Running, Role: LEADER
I20260811 18:57:09.451171 624 consensus_queue.cc:237] T 47a79343478f4d3a8c36d9f0e8a87e8d P 4610d1c6c23e4fae9be00c51787db5e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4610d1c6c23e4fae9be00c51787db5e1" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 36717 } } peers { permanent_uuid: "0273b8a84b2144519add9ca5bff112c8" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33685 } } peers { permanent_uuid: "dbe1e575579f4a6d86cb269f8ba75686" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45795 } }
I20260811 18:57:09.455245 32171 catalog_manager.cc:5495] T 47a79343478f4d3a8c36d9f0e8a87e8d P 4610d1c6c23e4fae9be00c51787db5e1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4610d1c6c23e4fae9be00c51787db5e1 (127.29.112.129). New cstate: current_term: 1 leader_uuid: "4610d1c6c23e4fae9be00c51787db5e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4610d1c6c23e4fae9be00c51787db5e1" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 36717 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0273b8a84b2144519add9ca5bff112c8" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbe1e575579f4a6d86cb269f8ba75686" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45795 } health_report { overall_health: UNKNOWN } } }
Aug 11 18:57:09 dist-test-slave-h6n0 krb5kdc[32090](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474626, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.190@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 18:57:09.517213 687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:09.497653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40010 (local address 127.29.112.190:46821)
0811 18:57:09.498333 (+ 680us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:09.498340 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:09.498738 (+ 398us) server_negotiation.cc:407] Connection header received
0811 18:57:09.498956 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:09.498966 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:09.499022 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:09.499143 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:09.500752 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:09.501572 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:09.502951 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:09.503431 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:09.503581 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:09.510067 (+ 6486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:09.510073 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:09.510077 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:09.510113 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:09.513680 (+ 3567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:09.514632 (+ 952us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:09.514643 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:09.514647 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:09.514796 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:09.515215 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:09.515225 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:09.515227 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:09.515552 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:09.515729 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:09.516367 (+ 638us) server_negotiation.cc:299] Negotiation successful
0811 18:57:09.516786 (+ 419us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":121,"server-negotiator.queue_time_us":515,"thread_start_us":357,"threads_started":1}
I20260811 18:57:09.517237 686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:09.497624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:46821 (local address 127.0.0.1:40010)
0811 18:57:09.498623 (+ 999us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:09.498640 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:09.498809 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:09.499169 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:09.499174 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:09.500359 (+ 1185us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:57:09.500590 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:09.500601 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:09.501715 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:09.501720 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:09.502805 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:09.502818 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:09.504013 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:09.504030 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:09.504455 (+ 425us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:09.505536 (+ 1081us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:57:09.505576 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:57:09.509759 (+ 4183us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:57:09.513883 (+ 4124us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:09.513902 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:09.513913 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:09.514456 (+ 543us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:09.514916 (+ 460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:09.514923 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:09.514926 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:09.515088 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:09.515711 (+ 623us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:57:09.515721 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:57:09.515903 (+ 182us) client_negotiation.cc:769] Sending connection context
0811 18:57:09.516087 (+ 184us) client_negotiation.cc:240] Negotiation successful
0811 18:57:09.516614 (+ 527us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":805,"thread_start_us":649,"threads_started":1}
I20260811 18:57:09.524386 32171 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.29.112.190@KRBTEST.COM'} at 127.0.0.1:40010:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 18:57:09.525548 32171 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:09.533200 32667 tablet_service.cc:1467] Processing CreateTablet for tablet 228baa4a94df4291a91e0cbe303e118f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=35a4054123a94b7e8c32acc7f8257144]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 18:57:09.533177 32414 tablet_service.cc:1467] Processing CreateTablet for tablet 228baa4a94df4291a91e0cbe303e118f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=35a4054123a94b7e8c32acc7f8257144]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 18:57:09.533756 32667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 228baa4a94df4291a91e0cbe303e118f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:09.533824 32414 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 228baa4a94df4291a91e0cbe303e118f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:09.538300 455 tablet_service.cc:1467] Processing CreateTablet for tablet 228baa4a94df4291a91e0cbe303e118f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=35a4054123a94b7e8c32acc7f8257144]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 18:57:09.538941 455 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 228baa4a94df4291a91e0cbe303e118f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:09.541329 613 tablet_bootstrap.cc:492] T 228baa4a94df4291a91e0cbe303e118f P 0273b8a84b2144519add9ca5bff112c8: Bootstrap starting.
I20260811 18:57:09.541800 620 tablet_bootstrap.cc:492] T 228baa4a94df4291a91e0cbe303e118f P 4610d1c6c23e4fae9be00c51787db5e1: Bootstrap starting.
I20260811 18:57:09.543157 613 tablet_bootstrap.cc:654] T 228baa4a94df4291a91e0cbe303e118f P 0273b8a84b2144519add9ca5bff112c8: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:09.544232 620 tablet_bootstrap.cc:654] T 228baa4a94df4291a91e0cbe303e118f P 4610d1c6c23e4fae9be00c51787db5e1: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:09.551025 615 tablet_bootstrap.cc:492] T 228baa4a94df4291a91e0cbe303e118f P dbe1e575579f4a6d86cb269f8ba75686: Bootstrap starting.
I20260811 18:57:09.553557 615 tablet_bootstrap.cc:654] T 228baa4a94df4291a91e0cbe303e118f P dbe1e575579f4a6d86cb269f8ba75686: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:09.554740 620 tablet_bootstrap.cc:492] T 228baa4a94df4291a91e0cbe303e118f P 4610d1c6c23e4fae9be00c51787db5e1: No bootstrap required, opened a new log
I20260811 18:57:09.554953 620 ts_tablet_manager.cc:1397] T 228baa4a94df4291a91e0cbe303e118f P 4610d1c6c23e4fae9be00c51787db5e1: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.011s
I20260811 18:57:09.555881 620 raft_consensus.cc:348] T 228baa4a94df4291a91e0cbe303e118f P 4610d1c6c23e4fae9be00c51787db5e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4610d1c6c23e4fae9be00c51787db5e1" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 36717 } } peers { permanent_uuid: "0273b8a84b2144519add9ca5bff112c8" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33685 } } peers { permanent_uuid: "dbe1e575579f4a6d86cb269f8ba75686" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45795 } }
I20260811 18:57:09.556161 620 raft_consensus.cc:374] T 228baa4a94df4291a91e0cbe303e118f P 4610d1c6c23e4fae9be00c51787db5e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:09.556250 620 raft_consensus.cc:729] T 228baa4a94df4291a91e0cbe303e118f P 4610d1c6c23e4fae9be00c51787db5e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4610d1c6c23e4fae9be00c51787db5e1, State: Initialized, Role: FOLLOWER
I20260811 18:57:09.556541 620 consensus_queue.cc:260] T 228baa4a94df4291a91e0cbe303e118f P 4610d1c6c23e4fae9be00c51787db5e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4610d1c6c23e4fae9be00c51787db5e1" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 36717 } } peers { permanent_uuid: "0273b8a84b2144519add9ca5bff112c8" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33685 } } peers { permanent_uuid: "dbe1e575579f4a6d86cb269f8ba75686" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45795 } }
I20260811 18:57:09.557060 624 tablet_replica.cc:439] T 228baa4a94df4291a91e0cbe303e118f P 4610d1c6c23e4fae9be00c51787db5e1: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4610d1c6c23e4fae9be00c51787db5e1" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 36717 } } peers { permanent_uuid: "0273b8a84b2144519add9ca5bff112c8" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33685 } } peers { permanent_uuid: "dbe1e575579f4a6d86cb269f8ba75686" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45795 } } }
I20260811 18:57:09.557353 624 tablet_replica.cc:442] T 228baa4a94df4291a91e0cbe303e118f P 4610d1c6c23e4fae9be00c51787db5e1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:57:09.558810 620 ts_tablet_manager.cc:1428] T 228baa4a94df4291a91e0cbe303e118f P 4610d1c6c23e4fae9be00c51787db5e1: Time spent starting tablet: real 0.004s user 0.002s sys 0.001s
I20260811 18:57:09.560791 613 tablet_bootstrap.cc:492] T 228baa4a94df4291a91e0cbe303e118f P 0273b8a84b2144519add9ca5bff112c8: No bootstrap required, opened a new log
I20260811 18:57:09.560992 613 ts_tablet_manager.cc:1397] T 228baa4a94df4291a91e0cbe303e118f P 0273b8a84b2144519add9ca5bff112c8: Time spent bootstrapping tablet: real 0.020s user 0.010s sys 0.007s
I20260811 18:57:09.561830 613 raft_consensus.cc:348] T 228baa4a94df4291a91e0cbe303e118f P 0273b8a84b2144519add9ca5bff112c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4610d1c6c23e4fae9be00c51787db5e1" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 36717 } } peers { permanent_uuid: "0273b8a84b2144519add9ca5bff112c8" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33685 } } peers { permanent_uuid: "dbe1e575579f4a6d86cb269f8ba75686" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45795 } }
I20260811 18:57:09.562116 613 raft_consensus.cc:374] T 228baa4a94df4291a91e0cbe303e118f P 0273b8a84b2144519add9ca5bff112c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:09.562207 613 raft_consensus.cc:729] T 228baa4a94df4291a91e0cbe303e118f P 0273b8a84b2144519add9ca5bff112c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0273b8a84b2144519add9ca5bff112c8, State: Initialized, Role: FOLLOWER
I20260811 18:57:09.562561 613 consensus_queue.cc:260] T 228baa4a94df4291a91e0cbe303e118f P 0273b8a84b2144519add9ca5bff112c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4610d1c6c23e4fae9be00c51787db5e1" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 36717 } } peers { permanent_uuid: "0273b8a84b2144519add9ca5bff112c8" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33685 } } peers { permanent_uuid: "dbe1e575579f4a6d86cb269f8ba75686" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45795 } }
I20260811 18:57:09.563177 622 tablet_replica.cc:439] T 228baa4a94df4291a91e0cbe303e118f P 0273b8a84b2144519add9ca5bff112c8: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4610d1c6c23e4fae9be00c51787db5e1" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 36717 } } peers { permanent_uuid: "0273b8a84b2144519add9ca5bff112c8" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33685 } } peers { permanent_uuid: "dbe1e575579f4a6d86cb269f8ba75686" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45795 } } }
I20260811 18:57:09.563675 622 tablet_replica.cc:442] T 228baa4a94df4291a91e0cbe303e118f P 0273b8a84b2144519add9ca5bff112c8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:57:09.564379 613 ts_tablet_manager.cc:1428] T 228baa4a94df4291a91e0cbe303e118f P 0273b8a84b2144519add9ca5bff112c8: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 18:57:09.569665 615 tablet_bootstrap.cc:492] T 228baa4a94df4291a91e0cbe303e118f P dbe1e575579f4a6d86cb269f8ba75686: No bootstrap required, opened a new log
I20260811 18:57:09.569911 615 ts_tablet_manager.cc:1397] T 228baa4a94df4291a91e0cbe303e118f P dbe1e575579f4a6d86cb269f8ba75686: Time spent bootstrapping tablet: real 0.019s user 0.007s sys 0.004s
I20260811 18:57:09.570751 615 raft_consensus.cc:348] T 228baa4a94df4291a91e0cbe303e118f P dbe1e575579f4a6d86cb269f8ba75686 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4610d1c6c23e4fae9be00c51787db5e1" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 36717 } } peers { permanent_uuid: "0273b8a84b2144519add9ca5bff112c8" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33685 } } peers { permanent_uuid: "dbe1e575579f4a6d86cb269f8ba75686" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45795 } }
I20260811 18:57:09.571003 615 raft_consensus.cc:374] T 228baa4a94df4291a91e0cbe303e118f P dbe1e575579f4a6d86cb269f8ba75686 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:09.571130 615 raft_consensus.cc:729] T 228baa4a94df4291a91e0cbe303e118f P dbe1e575579f4a6d86cb269f8ba75686 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbe1e575579f4a6d86cb269f8ba75686, State: Initialized, Role: FOLLOWER
I20260811 18:57:09.571416 615 consensus_queue.cc:260] T 228baa4a94df4291a91e0cbe303e118f P dbe1e575579f4a6d86cb269f8ba75686 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4610d1c6c23e4fae9be00c51787db5e1" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 36717 } } peers { permanent_uuid: "0273b8a84b2144519add9ca5bff112c8" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33685 } } peers { permanent_uuid: "dbe1e575579f4a6d86cb269f8ba75686" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45795 } }
I20260811 18:57:09.572124 615 ts_tablet_manager.cc:1428] T 228baa4a94df4291a91e0cbe303e118f P dbe1e575579f4a6d86cb269f8ba75686: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260811 18:57:09.571933 628 tablet_replica.cc:439] T 228baa4a94df4291a91e0cbe303e118f P dbe1e575579f4a6d86cb269f8ba75686: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4610d1c6c23e4fae9be00c51787db5e1" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 36717 } } peers { permanent_uuid: "0273b8a84b2144519add9ca5bff112c8" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33685 } } peers { permanent_uuid: "dbe1e575579f4a6d86cb269f8ba75686" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45795 } } }
I20260811 18:57:09.572324 628 tablet_replica.cc:442] T 228baa4a94df4291a91e0cbe303e118f P dbe1e575579f4a6d86cb269f8ba75686: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:57:09.606669 622 raft_consensus.cc:482] T 228baa4a94df4291a91e0cbe303e118f P 0273b8a84b2144519add9ca5bff112c8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:09.606956 622 raft_consensus.cc:504] T 228baa4a94df4291a91e0cbe303e118f P 0273b8a84b2144519add9ca5bff112c8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4610d1c6c23e4fae9be00c51787db5e1" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 36717 } } peers { permanent_uuid: "0273b8a84b2144519add9ca5bff112c8" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33685 } } peers { permanent_uuid: "dbe1e575579f4a6d86cb269f8ba75686" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45795 } }
I20260811 18:57:09.609066 622 leader_election.cc:288] T 228baa4a94df4291a91e0cbe303e118f P 0273b8a84b2144519add9ca5bff112c8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4610d1c6c23e4fae9be00c51787db5e1 (127.29.112.129:36717), dbe1e575579f4a6d86cb269f8ba75686 (127.29.112.131:45795)
I20260811 18:57:09.635145 607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:09.609710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.130:33643 (local address 127.29.112.131:45795)
0811 18:57:09.609952 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:09.609960 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:09.625698 (+ 15738us) server_negotiation.cc:407] Connection header received
0811 18:57:09.626718 (+ 1020us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:09.626723 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:09.626804 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:09.626889 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:57:09.627763 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:09.628910 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:09.632191 (+ 3281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:09.633733 (+ 1542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:09.633881 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:09.633957 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:09.634628 (+ 671us) server_negotiation.cc:299] Negotiation successful
0811 18:57:09.634787 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 18:57:09.636281 693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:09.609834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.131:45795 (local address 127.29.112.130:33643)
0811 18:57:09.625533 (+ 15699us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:09.625568 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:09.626519 (+ 951us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:09.627019 (+ 500us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:09.627023 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:09.627070 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:57:09.627499 (+ 429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:09.627506 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:09.629105 (+ 1599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:09.629110 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:09.631982 (+ 2872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:09.632003 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:09.634175 (+ 2172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:09.634182 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:09.634385 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:09.634421 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 18:57:09.635674 (+ 1253us) client_negotiation.cc:240] Negotiation successful
0811 18:57:09.635841 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15486,"mutex_wait_us":32,"thread_start_us":16451,"threads_started":1}
I20260811 18:57:09.636929 694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:09.610174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:36717 (local address 127.29.112.130:58165)
0811 18:57:09.626234 (+ 16060us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:09.626252 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:09.626415 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:09.626862 (+ 447us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:09.626867 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:09.626899 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:57:09.627418 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:09.627436 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:09.628742 (+ 1306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:09.628748 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:09.632105 (+ 3357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:09.632125 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:09.635838 (+ 3713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:09.635844 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:09.636014 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:09.636046 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 18:57:09.636138 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 18:57:09.636294 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15934,"mutex_wait_us":3,"thread_start_us":15727,"threads_started":1}
I20260811 18:57:09.637315 492 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "228baa4a94df4291a91e0cbe303e118f" candidate_uuid: "0273b8a84b2144519add9ca5bff112c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbe1e575579f4a6d86cb269f8ba75686" is_pre_election: true
I20260811 18:57:09.637609 492 raft_consensus.cc:2454] T 228baa4a94df4291a91e0cbe303e118f P dbe1e575579f4a6d86cb269f8ba75686 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0273b8a84b2144519add9ca5bff112c8 in term 0.
I20260811 18:57:09.638043 692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:09.609922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.130:58165 (local address 127.29.112.129:36717)
0811 18:57:09.626196 (+ 16274us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:09.626204 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:09.626369 (+ 165us) server_negotiation.cc:407] Connection header received
0811 18:57:09.626598 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:09.626602 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:09.626708 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:09.626811 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:57:09.627651 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:09.628576 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:09.632324 (+ 3748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:09.635328 (+ 3004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:09.635532 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:09.635609 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:09.637344 (+ 1735us) server_negotiation.cc:299] Negotiation successful
0811 18:57:09.637524 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16135,"thread_start_us":16270,"threads_started":1}
I20260811 18:57:09.638813 32579 leader_election.cc:302] T 228baa4a94df4291a91e0cbe303e118f P 0273b8a84b2144519add9ca5bff112c8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0273b8a84b2144519add9ca5bff112c8, dbe1e575579f4a6d86cb269f8ba75686; no voters:
I20260811 18:57:09.638935 32453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "228baa4a94df4291a91e0cbe303e118f" candidate_uuid: "0273b8a84b2144519add9ca5bff112c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4610d1c6c23e4fae9be00c51787db5e1" is_pre_election: true
I20260811 18:57:09.639192 32453 raft_consensus.cc:2454] T 228baa4a94df4291a91e0cbe303e118f P 4610d1c6c23e4fae9be00c51787db5e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0273b8a84b2144519add9ca5bff112c8 in term 0.
I20260811 18:57:09.639355 622 raft_consensus.cc:2781] T 228baa4a94df4291a91e0cbe303e118f P 0273b8a84b2144519add9ca5bff112c8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:09.639489 622 raft_consensus.cc:482] T 228baa4a94df4291a91e0cbe303e118f P 0273b8a84b2144519add9ca5bff112c8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:09.639570 622 raft_consensus.cc:3037] T 228baa4a94df4291a91e0cbe303e118f P 0273b8a84b2144519add9ca5bff112c8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:09.642889 622 raft_consensus.cc:504] T 228baa4a94df4291a91e0cbe303e118f P 0273b8a84b2144519add9ca5bff112c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4610d1c6c23e4fae9be00c51787db5e1" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 36717 } } peers { permanent_uuid: "0273b8a84b2144519add9ca5bff112c8" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33685 } } peers { permanent_uuid: "dbe1e575579f4a6d86cb269f8ba75686" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45795 } }
I20260811 18:57:09.643630 622 leader_election.cc:288] T 228baa4a94df4291a91e0cbe303e118f P 0273b8a84b2144519add9ca5bff112c8 [CANDIDATE]: Term 1 election: Requested vote from peers 4610d1c6c23e4fae9be00c51787db5e1 (127.29.112.129:36717), dbe1e575579f4a6d86cb269f8ba75686 (127.29.112.131:45795)
I20260811 18:57:09.644248 492 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "228baa4a94df4291a91e0cbe303e118f" candidate_uuid: "0273b8a84b2144519add9ca5bff112c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbe1e575579f4a6d86cb269f8ba75686"
I20260811 18:57:09.644241 32453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "228baa4a94df4291a91e0cbe303e118f" candidate_uuid: "0273b8a84b2144519add9ca5bff112c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4610d1c6c23e4fae9be00c51787db5e1"
I20260811 18:57:09.644501 492 raft_consensus.cc:3037] T 228baa4a94df4291a91e0cbe303e118f P dbe1e575579f4a6d86cb269f8ba75686 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:09.644515 32453 raft_consensus.cc:3037] T 228baa4a94df4291a91e0cbe303e118f P 4610d1c6c23e4fae9be00c51787db5e1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:09.648404 492 raft_consensus.cc:2454] T 228baa4a94df4291a91e0cbe303e118f P dbe1e575579f4a6d86cb269f8ba75686 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0273b8a84b2144519add9ca5bff112c8 in term 1.
I20260811 18:57:09.648504 32453 raft_consensus.cc:2454] T 228baa4a94df4291a91e0cbe303e118f P 4610d1c6c23e4fae9be00c51787db5e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0273b8a84b2144519add9ca5bff112c8 in term 1.
I20260811 18:57:09.649122 32575 leader_election.cc:302] T 228baa4a94df4291a91e0cbe303e118f P 0273b8a84b2144519add9ca5bff112c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0273b8a84b2144519add9ca5bff112c8, 4610d1c6c23e4fae9be00c51787db5e1; no voters:
I20260811 18:57:09.649683 622 raft_consensus.cc:2781] T 228baa4a94df4291a91e0cbe303e118f P 0273b8a84b2144519add9ca5bff112c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:09.650251 622 raft_consensus.cc:686] T 228baa4a94df4291a91e0cbe303e118f P 0273b8a84b2144519add9ca5bff112c8 [term 1 LEADER]: Becoming Leader. State: Replica: 0273b8a84b2144519add9ca5bff112c8, State: Running, Role: LEADER
I20260811 18:57:09.650760 622 consensus_queue.cc:237] T 228baa4a94df4291a91e0cbe303e118f P 0273b8a84b2144519add9ca5bff112c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4610d1c6c23e4fae9be00c51787db5e1" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 36717 } } peers { permanent_uuid: "0273b8a84b2144519add9ca5bff112c8" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33685 } } peers { permanent_uuid: "dbe1e575579f4a6d86cb269f8ba75686" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45795 } }
I20260811 18:57:09.653427 697 tablet_replica.cc:439] T 228baa4a94df4291a91e0cbe303e118f P 0273b8a84b2144519add9ca5bff112c8: TxnStatusTablet state changed. Reason: New leader 0273b8a84b2144519add9ca5bff112c8. Latest consensus state: current_term: 1 leader_uuid: "0273b8a84b2144519add9ca5bff112c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4610d1c6c23e4fae9be00c51787db5e1" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 36717 } } peers { permanent_uuid: "0273b8a84b2144519add9ca5bff112c8" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33685 } } peers { permanent_uuid: "dbe1e575579f4a6d86cb269f8ba75686" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45795 } } }
I20260811 18:57:09.654479 697 tablet_replica.cc:442] T 228baa4a94df4291a91e0cbe303e118f P 0273b8a84b2144519add9ca5bff112c8: This TxnStatusTablet replica's current role is: LEADER
I20260811 18:57:09.655543 700 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 18:57:09.656083 700 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 18:57:09.656204 32171 catalog_manager.cc:5495] T 228baa4a94df4291a91e0cbe303e118f P 0273b8a84b2144519add9ca5bff112c8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0273b8a84b2144519add9ca5bff112c8 (127.29.112.130). New cstate: current_term: 1 leader_uuid: "0273b8a84b2144519add9ca5bff112c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4610d1c6c23e4fae9be00c51787db5e1" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 36717 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0273b8a84b2144519add9ca5bff112c8" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe1e575579f4a6d86cb269f8ba75686" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45795 } health_report { overall_health: UNKNOWN } } }
I20260811 18:57:09.693037 605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:09.686742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37144 (local address 127.29.112.130:33685)
0811 18:57:09.686935 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:09.686941 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:09.687218 (+ 277us) server_negotiation.cc:407] Connection header received
0811 18:57:09.687513 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:09.687527 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:09.687645 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:09.687789 (+ 144us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:57:09.688168 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:09.689022 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:09.690746 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:09.691202 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:09.691344 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:09.691810 (+ 466us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:57:09.692341 (+ 531us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:57:09.692493 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:09.692604 (+ 111us) server_negotiation.cc:299] Negotiation successful
0811 18:57:09.692685 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 18:57:09.693408 686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:09.686749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.130:33685 (local address 127.0.0.1:37144)
0811 18:57:09.687119 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:09.687135 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:09.687309 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:09.687813 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:09.687817 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:09.687829 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 18:57:09.688011 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:09.688027 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:09.689214 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:09.689221 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:09.690608 (+ 1387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:09.690620 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:09.691368 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:09.691373 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:09.691558 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:09.691615 (+ 57us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 18:57:09.692454 (+ 839us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 18:57:09.692474 (+ 20us) client_negotiation.cc:769] Sending connection context
0811 18:57:09.692569 (+ 95us) client_negotiation.cc:240] Negotiation successful
0811 18:57:09.692725 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217}
I20260811 18:57:09.699133 32453 raft_consensus.cc:1261] T 228baa4a94df4291a91e0cbe303e118f P 4610d1c6c23e4fae9be00c51787db5e1 [term 1 FOLLOWER]: Refusing update from remote peer 0273b8a84b2144519add9ca5bff112c8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:57:09.699281 492 raft_consensus.cc:1261] T 228baa4a94df4291a91e0cbe303e118f P dbe1e575579f4a6d86cb269f8ba75686 [term 1 FOLLOWER]: Refusing update from remote peer 0273b8a84b2144519add9ca5bff112c8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:57:09.699967 697 consensus_queue.cc:1035] T 228baa4a94df4291a91e0cbe303e118f P 0273b8a84b2144519add9ca5bff112c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4610d1c6c23e4fae9be00c51787db5e1" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 36717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:09.700269 622 consensus_queue.cc:1035] T 228baa4a94df4291a91e0cbe303e118f P 0273b8a84b2144519add9ca5bff112c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe1e575579f4a6d86cb269f8ba75686" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:09.714764 624 tablet_replica.cc:439] T 228baa4a94df4291a91e0cbe303e118f P 4610d1c6c23e4fae9be00c51787db5e1: TxnStatusTablet state changed. Reason: New leader 0273b8a84b2144519add9ca5bff112c8. Latest consensus state: current_term: 1 leader_uuid: "0273b8a84b2144519add9ca5bff112c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4610d1c6c23e4fae9be00c51787db5e1" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 36717 } } peers { permanent_uuid: "0273b8a84b2144519add9ca5bff112c8" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33685 } } peers { permanent_uuid: "dbe1e575579f4a6d86cb269f8ba75686" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45795 } } }
I20260811 18:57:09.715325 624 tablet_replica.cc:442] T 228baa4a94df4291a91e0cbe303e118f P 4610d1c6c23e4fae9be00c51787db5e1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:57:09.720474 622 tablet_replica.cc:439] T 228baa4a94df4291a91e0cbe303e118f P 0273b8a84b2144519add9ca5bff112c8: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0273b8a84b2144519add9ca5bff112c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4610d1c6c23e4fae9be00c51787db5e1" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 36717 } } peers { permanent_uuid: "0273b8a84b2144519add9ca5bff112c8" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33685 } } peers { permanent_uuid: "dbe1e575579f4a6d86cb269f8ba75686" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45795 } } }
I20260811 18:57:09.720808 622 tablet_replica.cc:442] T 228baa4a94df4291a91e0cbe303e118f P 0273b8a84b2144519add9ca5bff112c8: This TxnStatusTablet replica's current role is: LEADER
I20260811 18:57:09.721068 709 mvcc.cc:205] Tried to move back new op lower bound from 7317400083239596032 to 7317400083054895104. Current Snapshot: MvccSnapshot[applied={T|T < 7317400083239596032}]
I20260811 18:57:09.724718 708 mvcc.cc:205] Tried to move back new op lower bound from 7317400083239596032 to 7317400083054895104. Current Snapshot: MvccSnapshot[applied={T|T < 7317400083239596032}]
I20260811 18:57:09.730917 624 tablet_replica.cc:439] T 228baa4a94df4291a91e0cbe303e118f P 4610d1c6c23e4fae9be00c51787db5e1: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0273b8a84b2144519add9ca5bff112c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4610d1c6c23e4fae9be00c51787db5e1" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 36717 } } peers { permanent_uuid: "0273b8a84b2144519add9ca5bff112c8" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33685 } } peers { permanent_uuid: "dbe1e575579f4a6d86cb269f8ba75686" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45795 } } }
I20260811 18:57:09.731333 624 tablet_replica.cc:442] T 228baa4a94df4291a91e0cbe303e118f P 4610d1c6c23e4fae9be00c51787db5e1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:57:09.735564 628 tablet_replica.cc:439] T 228baa4a94df4291a91e0cbe303e118f P dbe1e575579f4a6d86cb269f8ba75686: TxnStatusTablet state changed. Reason: New leader 0273b8a84b2144519add9ca5bff112c8. Latest consensus state: current_term: 1 leader_uuid: "0273b8a84b2144519add9ca5bff112c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4610d1c6c23e4fae9be00c51787db5e1" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 36717 } } peers { permanent_uuid: "0273b8a84b2144519add9ca5bff112c8" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33685 } } peers { permanent_uuid: "dbe1e575579f4a6d86cb269f8ba75686" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45795 } } }
I20260811 18:57:09.735926 628 tablet_replica.cc:442] T 228baa4a94df4291a91e0cbe303e118f P dbe1e575579f4a6d86cb269f8ba75686: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:57:09.739892 697 tablet_replica.cc:439] T 228baa4a94df4291a91e0cbe303e118f P 0273b8a84b2144519add9ca5bff112c8: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0273b8a84b2144519add9ca5bff112c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4610d1c6c23e4fae9be00c51787db5e1" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 36717 } } peers { permanent_uuid: "0273b8a84b2144519add9ca5bff112c8" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33685 } } peers { permanent_uuid: "dbe1e575579f4a6d86cb269f8ba75686" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45795 } } }
I20260811 18:57:09.740309 697 tablet_replica.cc:442] T 228baa4a94df4291a91e0cbe303e118f P 0273b8a84b2144519add9ca5bff112c8: This TxnStatusTablet replica's current role is: LEADER
I20260811 18:57:09.755504 628 tablet_replica.cc:439] T 228baa4a94df4291a91e0cbe303e118f P dbe1e575579f4a6d86cb269f8ba75686: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0273b8a84b2144519add9ca5bff112c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4610d1c6c23e4fae9be00c51787db5e1" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 36717 } } peers { permanent_uuid: "0273b8a84b2144519add9ca5bff112c8" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33685 } } peers { permanent_uuid: "dbe1e575579f4a6d86cb269f8ba75686" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45795 } } }
I20260811 18:57:09.755811 628 tablet_replica.cc:442] T 228baa4a94df4291a91e0cbe303e118f P dbe1e575579f4a6d86cb269f8ba75686: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:57:09.764111 692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:09.744106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35302 (local address 127.29.112.129:36717)
0811 18:57:09.744291 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:09.744298 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:09.753539 (+ 9241us) server_negotiation.cc:407] Connection header received
0811 18:57:09.753915 (+ 376us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:09.753926 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:09.754111 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:09.754336 (+ 225us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:57:09.759345 (+ 5009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:09.760400 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:09.762007 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:09.762404 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:09.762589 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:09.762962 (+ 373us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:57:09.763472 (+ 510us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:57:09.763653 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:09.763714 (+ 61us) server_negotiation.cc:299] Negotiation successful
0811 18:57:09.763801 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 18:57:09.781756 605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:09.775377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.129:59237 (local address 127.29.112.130:33685)
0811 18:57:09.775819 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:09.775826 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:09.776217 (+ 391us) server_negotiation.cc:407] Connection header received
0811 18:57:09.776470 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:09.776476 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:09.776600 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:09.776731 (+ 131us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:57:09.777488 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:09.778460 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:09.779914 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:09.780401 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:09.780551 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:09.780994 (+ 443us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:57:09.781099 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:57:09.781200 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:09.781335 (+ 135us) server_negotiation.cc:299] Negotiation successful
0811 18:57:09.781412 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":244}
I20260811 18:57:09.781831 732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:09.775381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.130:33685 (local address 127.29.112.129:59237)
0811 18:57:09.776088 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:09.776117 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:09.776309 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:09.777030 (+ 721us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:09.777034 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:09.777055 (+ 21us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 18:57:09.777304 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:09.777311 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:09.778686 (+ 1375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:09.778692 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:09.779764 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:09.779776 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:09.780589 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:09.780597 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:09.780736 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:09.780793 (+ 57us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 18:57:09.781203 (+ 410us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 18:57:09.781217 (+ 14us) client_negotiation.cc:769] Sending connection context
0811 18:57:09.781312 (+ 95us) client_negotiation.cc:240] Negotiation successful
0811 18:57:09.781448 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":452,"threads_started":1}
I20260811 18:57:09.793919 32707 raft_consensus.cc:1261] T 47a79343478f4d3a8c36d9f0e8a87e8d P 0273b8a84b2144519add9ca5bff112c8 [term 1 FOLLOWER]: Refusing update from remote peer 4610d1c6c23e4fae9be00c51787db5e1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:57:09.793919 492 raft_consensus.cc:1261] T 47a79343478f4d3a8c36d9f0e8a87e8d P dbe1e575579f4a6d86cb269f8ba75686 [term 1 FOLLOWER]: Refusing update from remote peer 4610d1c6c23e4fae9be00c51787db5e1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:57:09.794883 669 consensus_queue.cc:1035] T 47a79343478f4d3a8c36d9f0e8a87e8d P 4610d1c6c23e4fae9be00c51787db5e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe1e575579f4a6d86cb269f8ba75686" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:09.795548 669 consensus_queue.cc:1035] T 47a79343478f4d3a8c36d9f0e8a87e8d P 4610d1c6c23e4fae9be00c51787db5e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0273b8a84b2144519add9ca5bff112c8" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:09.840137 692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:09.831843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.130:35873 (local address 127.29.112.129:36717)
0811 18:57:09.832055 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:09.832062 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:09.832860 (+ 798us) server_negotiation.cc:407] Connection header received
0811 18:57:09.833109 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:09.833115 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:09.833234 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:09.833358 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:57:09.834095 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:09.835286 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:09.837490 (+ 2204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:09.838431 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:09.838582 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:09.839257 (+ 675us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:57:09.839410 (+ 153us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:57:09.839595 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:09.839755 (+ 160us) server_negotiation.cc:299] Negotiation successful
0811 18:57:09.839818 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 18:57:09.840342 737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:09.831898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:36717 (local address 127.29.112.130:35873)
0811 18:57:09.832728 (+ 830us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:09.832757 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:09.832947 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:09.833403 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:09.833408 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:09.833424 (+ 16us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 18:57:09.833900 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:09.833917 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:09.835429 (+ 1512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:09.835434 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:09.837277 (+ 1843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:09.837300 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:09.838775 (+ 1475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:09.838783 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:09.838990 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:09.839066 (+ 76us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 18:57:09.839631 (+ 565us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 18:57:09.839643 (+ 12us) client_negotiation.cc:769] Sending connection context
0811 18:57:09.839741 (+ 98us) client_negotiation.cc:240] Negotiation successful
0811 18:57:09.839894 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":616,"thread_start_us":417,"threads_started":1}
I20260811 18:57:09.890847 32171 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38934:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:57:09.891148 32171 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38934:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:57:09.894749 32171 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 679be82e1e23435fb4820f5c2b5cac04: Sending DeleteTablet for 3 replicas of tablet 47a79343478f4d3a8c36d9f0e8a87e8d
I20260811 18:57:09.896116 32667 tablet_service.cc:1514] Processing DeleteTablet for tablet 47a79343478f4d3a8c36d9f0e8a87e8d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:09 UTC) from {username='kudu', principal='kudu/127.29.112.190@KRBTEST.COM'} at 127.0.0.1:43336
I20260811 18:57:09.896121 455 tablet_service.cc:1514] Processing DeleteTablet for tablet 47a79343478f4d3a8c36d9f0e8a87e8d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:09 UTC) from {username='kudu', principal='kudu/127.29.112.190@KRBTEST.COM'} at 127.0.0.1:54480
I20260811 18:57:09.896719 32414 tablet_service.cc:1514] Processing DeleteTablet for tablet 47a79343478f4d3a8c36d9f0e8a87e8d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:09 UTC) from {username='kudu', principal='kudu/127.29.112.190@KRBTEST.COM'} at 127.0.0.1:35384
I20260811 18:57:09.908914 748 tablet_replica.cc:330] T 47a79343478f4d3a8c36d9f0e8a87e8d P dbe1e575579f4a6d86cb269f8ba75686: stopping tablet replica
I20260811 18:57:09.909140 748 raft_consensus.cc:2229] T 47a79343478f4d3a8c36d9f0e8a87e8d P dbe1e575579f4a6d86cb269f8ba75686 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:09.909397 748 raft_consensus.cc:2258] T 47a79343478f4d3a8c36d9f0e8a87e8d P dbe1e575579f4a6d86cb269f8ba75686 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:09.910962 748 ts_tablet_manager.cc:1905] T 47a79343478f4d3a8c36d9f0e8a87e8d P dbe1e575579f4a6d86cb269f8ba75686: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:09.920028 748 ts_tablet_manager.cc:1918] T 47a79343478f4d3a8c36d9f0e8a87e8d P dbe1e575579f4a6d86cb269f8ba75686: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 18:57:09.920199 748 log.cc:1196] T 47a79343478f4d3a8c36d9f0e8a87e8d P dbe1e575579f4a6d86cb269f8ba75686: Deleting WAL directory at /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/minicluster-data/ts-2/wal/wals/47a79343478f4d3a8c36d9f0e8a87e8d
I20260811 18:57:09.920761 748 ts_tablet_manager.cc:1939] T 47a79343478f4d3a8c36d9f0e8a87e8d P dbe1e575579f4a6d86cb269f8ba75686: Deleting consensus metadata
I20260811 18:57:09.922037 32146 catalog_manager.cc:4841] TS dbe1e575579f4a6d86cb269f8ba75686 (127.29.112.131:45795): tablet 47a79343478f4d3a8c36d9f0e8a87e8d (table test-table [id=44364c171fd2480dbc8637972ba88bb2]) successfully deleted
I20260811 18:57:09.931468 753 tablet_replica.cc:330] T 47a79343478f4d3a8c36d9f0e8a87e8d P 4610d1c6c23e4fae9be00c51787db5e1: stopping tablet replica
I20260811 18:57:09.931725 753 raft_consensus.cc:2229] T 47a79343478f4d3a8c36d9f0e8a87e8d P 4610d1c6c23e4fae9be00c51787db5e1 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:09.932714 753 raft_consensus.cc:2258] T 47a79343478f4d3a8c36d9f0e8a87e8d P 4610d1c6c23e4fae9be00c51787db5e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:09.935040 753 ts_tablet_manager.cc:1905] T 47a79343478f4d3a8c36d9f0e8a87e8d P 4610d1c6c23e4fae9be00c51787db5e1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:09.945097 753 ts_tablet_manager.cc:1918] T 47a79343478f4d3a8c36d9f0e8a87e8d P 4610d1c6c23e4fae9be00c51787db5e1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 18:57:09.945271 753 log.cc:1196] T 47a79343478f4d3a8c36d9f0e8a87e8d P 4610d1c6c23e4fae9be00c51787db5e1: Deleting WAL directory at /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/minicluster-data/ts-0/wal/wals/47a79343478f4d3a8c36d9f0e8a87e8d
I20260811 18:57:09.945758 753 ts_tablet_manager.cc:1939] T 47a79343478f4d3a8c36d9f0e8a87e8d P 4610d1c6c23e4fae9be00c51787db5e1: Deleting consensus metadata
Aug 11 18:57:09 dist-test-slave-h6n0 krb5kdc[32090](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474628, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.29.112.129@KRBTEST.COM
I20260811 18:57:09.946866 752 tablet_replica.cc:330] T 47a79343478f4d3a8c36d9f0e8a87e8d P 0273b8a84b2144519add9ca5bff112c8: stopping tablet replica
I20260811 18:57:09.947098 752 raft_consensus.cc:2229] T 47a79343478f4d3a8c36d9f0e8a87e8d P 0273b8a84b2144519add9ca5bff112c8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:09.947395 752 raft_consensus.cc:2258] T 47a79343478f4d3a8c36d9f0e8a87e8d P 0273b8a84b2144519add9ca5bff112c8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:09.947546 32144 catalog_manager.cc:4841] TS 4610d1c6c23e4fae9be00c51787db5e1 (127.29.112.129:36717): tablet 47a79343478f4d3a8c36d9f0e8a87e8d (table test-table [id=44364c171fd2480dbc8637972ba88bb2]) successfully deleted
I20260811 18:57:09.948783 752 ts_tablet_manager.cc:1905] T 47a79343478f4d3a8c36d9f0e8a87e8d P 0273b8a84b2144519add9ca5bff112c8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:09.957006 692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:09.932073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35314 (local address 127.29.112.129:36717)
0811 18:57:09.932261 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:09.932268 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:09.934331 (+ 2063us) server_negotiation.cc:407] Connection header received
0811 18:57:09.934583 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:09.934591 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:09.934682 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:09.934784 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:09.937031 (+ 2247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:09.938331 (+ 1300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:09.940902 (+ 2571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:09.941644 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:09.941741 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:09.948175 (+ 6434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:09.948192 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:09.948212 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:09.948258 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:09.952690 (+ 4432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:09.953783 (+ 1093us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:09.953793 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:09.953796 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:09.953998 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:09.954466 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:09.954471 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:09.954474 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:09.955390 (+ 916us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:09.955705 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:09.956368 (+ 663us) server_negotiation.cc:299] Negotiation successful
0811 18:57:09.956619 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 18:57:09.957422 752 ts_tablet_manager.cc:1918] T 47a79343478f4d3a8c36d9f0e8a87e8d P 0273b8a84b2144519add9ca5bff112c8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 18:57:09.957573 752 log.cc:1196] T 47a79343478f4d3a8c36d9f0e8a87e8d P 0273b8a84b2144519add9ca5bff112c8: Deleting WAL directory at /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474615205622-30146-0/minicluster-data/ts-1/wal/wals/47a79343478f4d3a8c36d9f0e8a87e8d
I20260811 18:57:09.958132 752 ts_tablet_manager.cc:1939] T 47a79343478f4d3a8c36d9f0e8a87e8d P 0273b8a84b2144519add9ca5bff112c8: Deleting consensus metadata
W20260811 18:57:09.958952 32534 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35314
I20260811 18:57:09.960199 32144 catalog_manager.cc:4841] TS 0273b8a84b2144519add9ca5bff112c8 (127.29.112.130:33685): tablet 47a79343478f4d3a8c36d9f0e8a87e8d (table test-table [id=44364c171fd2480dbc8637972ba88bb2]) successfully deleted
I20260811 18:57:09.978868 687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:09.963239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40016 (local address 127.29.112.190:46821)
0811 18:57:09.963460 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:09.963467 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:09.964303 (+ 836us) server_negotiation.cc:407] Connection header received
0811 18:57:09.964509 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:09.964515 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:09.964590 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:09.964697 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:09.966019 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:09.966859 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:09.968171 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:09.968925 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:09.969342 (+ 417us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:09.972337 (+ 2995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:09.972345 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:09.972356 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:09.972392 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:09.975650 (+ 3258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:09.976482 (+ 832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:09.976488 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:09.976492 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:09.976637 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:09.977091 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:09.977096 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:09.977100 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:09.977501 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:09.977741 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:09.978275 (+ 534us) server_negotiation.cc:299] Negotiation successful
0811 18:57:09.978505 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
W20260811 18:57:09.987174 32171 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40016
I20260811 18:57:09.991698 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 32292
I20260811 18:57:10.021803 692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:09.993095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35316 (local address 127.29.112.129:36717)
0811 18:57:09.994393 (+ 1298us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:09.994402 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:10.007066 (+ 12664us) server_negotiation.cc:407] Connection header received
0811 18:57:10.007369 (+ 303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:10.007375 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:10.007500 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:10.007597 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:10.009000 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:10.009837 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:10.011435 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:10.011916 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:10.012060 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:10.014492 (+ 2432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:10.014497 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:10.014500 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:10.014579 (+ 79us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:10.018169 (+ 3590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:10.019369 (+ 1200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:10.019375 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:10.019379 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:10.019645 (+ 266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:10.020122 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:10.020127 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:10.020131 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:10.020503 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:10.020783 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:10.021132 (+ 349us) server_negotiation.cc:299] Negotiation successful
0811 18:57:10.021393 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1115}
W20260811 18:57:10.022428 32534 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35316
W20260811 18:57:10.023113 30146 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 32292: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 18:57:10.023305 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 32292
I20260811 18:57:10.063239 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 32547
Aug 11 18:57:10 dist-test-slave-h6n0 krb5kdc[32090](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474628, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.29.112.130@KRBTEST.COM
I20260811 18:57:10.084259 605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:10.064455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37154 (local address 127.29.112.130:33685)
0811 18:57:10.064679 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:10.064686 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:10.064812 (+ 126us) server_negotiation.cc:407] Connection header received
0811 18:57:10.065043 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:10.065047 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:10.065114 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:10.065239 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:10.066504 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:10.067841 (+ 1337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:10.069445 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:10.070137 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:10.070262 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:10.075667 (+ 5405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:10.075688 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:10.075708 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:10.075784 (+ 76us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:10.080241 (+ 4457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:10.081274 (+ 1033us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:10.081300 (+ 26us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:10.081304 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:10.081606 (+ 302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:10.082234 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:10.082240 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:10.082244 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:10.082813 (+ 569us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:10.083217 (+ 404us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:10.083544 (+ 327us) server_negotiation.cc:299] Negotiation successful
0811 18:57:10.083841 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
W20260811 18:57:10.085063 313 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37154
W20260811 18:57:10.085804 30146 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 32547: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 18:57:10.086030 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 32547
I20260811 18:57:10.112228 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 331
Aug 11 18:57:10 dist-test-slave-h6n0 krb5kdc[32090](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474628, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.29.112.131@KRBTEST.COM
I20260811 18:57:10.129287 607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:10.113033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43494 (local address 127.29.112.131:45795)
0811 18:57:10.113224 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:10.113232 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:10.113297 (+ 65us) server_negotiation.cc:407] Connection header received
0811 18:57:10.113497 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:10.113502 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:10.113584 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:10.113690 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:10.115031 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:10.115769 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:10.117012 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:10.117441 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:10.117577 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:10.122169 (+ 4592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:10.122183 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:10.122200 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:10.122236 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:10.126132 (+ 3896us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:10.127351 (+ 1219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:10.127359 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:10.127361 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:10.127513 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:10.127862 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:10.127866 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:10.127868 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:10.128278 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:10.128554 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:10.128825 (+ 271us) server_negotiation.cc:299] Negotiation successful
0811 18:57:10.129014 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
W20260811 18:57:10.130057 570 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43494
W20260811 18:57:10.130856 30146 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 331: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 18:57:10.130992 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 331
I20260811 18:57:10.166692 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 32107
I20260811 18:57:10.168094 32233 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:57:10.293484 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 32107
2026-08-11T18:57:10Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (5379 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:57:10 dist-test-slave-h6n0 krb5kdc[771](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:57:10 dist-test-slave-h6n0 krb5kdc[771](info): set up 2 sockets
Aug 11 18:57:10 dist-test-slave-h6n0 krb5kdc[771](info): commencing operation
krb5kdc: starting...
W20260811 18:57:11.731283 30146 mini_kdc.cc:121] Time spent starting KDC: real 1.384s user 0.004s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/krb5kdc/test-user.keytab.
Aug 11 18:57:11 dist-test-slave-h6n0 krb5kdc[771](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474631, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:57:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:57:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.112.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab.
Entry for principal kudu/127.29.112.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab.
I20260811 18:57:11.958599 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.190:39833
--webserver_interface=127.29.112.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab
--principal=kudu/127.29.112.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:39255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.190:39833
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:57:12.301115 794 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:57:12.301400 794 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:57:12.301560 794 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:57:12.309111 794 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:57:12.309214 794 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 18:57:12.309310 794 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:57:12.309414 794 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:57:12.309494 794 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:57:12.309618 794 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:57:12.319742 794 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:39255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.112.190:39833
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab
--principal=kudu/127.29.112.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.190:39833
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.112.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:57:12.320982 794 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:57:12.322266 794 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:57:12.334146 805 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:12.334298 809 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:12.334468 800 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:57:12.335173 794 server_base.cc:1034] running on GCE node
I20260811 18:57:12.335863 794 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:57:12.337090 794 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:57:12.338335 794 hybrid_clock.cc:648] HybridClock initialized: now 1786474632338262 us; error 35 us; skew 500 ppm
I20260811 18:57:12.338716 794 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:57:12 dist-test-slave-h6n0 krb5kdc[771](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474632, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:57:12.342267 794 init.cc:377] Logged in from keytab as kudu/127.29.112.190@KRBTEST.COM (short username kudu)
I20260811 18:57:12.344281 794 webserver.cc:460] Webserver started at http://127.29.112.190:41559/ using document root <none> and password file <none>
I20260811 18:57:12.345554 794 fs_manager.cc:359] Metadata directory not provided
I20260811 18:57:12.345680 794 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:57:12.346081 794 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:57:12.349550 794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/minicluster-data/master-0/data/instance:
uuid: "50c83e7e69974b90a46fe19124750a8d"
format_stamp: "Formatted at 2026-08-11 18:57:12 on dist-test-slave-h6n0"
server_key: "f08f140b42ff200cbb72697348a26bf9"
server_key_iv: "59f08545c7e94cdc8ff5592d320beeda"
server_key_version: "encryptionkey@0"
I20260811 18:57:12.350332 794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/minicluster-data/master-0/wal/instance:
uuid: "50c83e7e69974b90a46fe19124750a8d"
format_stamp: "Formatted at 2026-08-11 18:57:12 on dist-test-slave-h6n0"
server_key: "f08f140b42ff200cbb72697348a26bf9"
server_key_iv: "59f08545c7e94cdc8ff5592d320beeda"
server_key_version: "encryptionkey@0"
I20260811 18:57:12.356374 794 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 18:57:12.360525 821 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:12.361871 794 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260811 18:57:12.362072 794 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/minicluster-data/master-0/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/minicluster-data/master-0/wal
uuid: "50c83e7e69974b90a46fe19124750a8d"
format_stamp: "Formatted at 2026-08-11 18:57:12 on dist-test-slave-h6n0"
server_key: "f08f140b42ff200cbb72697348a26bf9"
server_key_iv: "59f08545c7e94cdc8ff5592d320beeda"
server_key_version: "encryptionkey@0"
I20260811 18:57:12.362301 794 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:57:12.381165 794 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:57:12.391571 794 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:57:12.392207 794 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:57:12.441890 794 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.190:39833
I20260811 18:57:12.441951 933 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.190:39833 every 8 connection(s)
I20260811 18:57:12.444461 794 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
I20260811 18:57:12.451872 934 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:12.453711 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 794
I20260811 18:57:12.454057 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/minicluster-data/master-0/wal/instance
I20260811 18:57:12.456249 30146 external_mini_cluster.cc:1428] Setting key daa53e2168d50a2691584359628841d3
I20260811 18:57:12.488544 934 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50c83e7e69974b90a46fe19124750a8d: Bootstrap starting.
Aug 11 18:57:12 dist-test-slave-h6n0 krb5kdc[771](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474631, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 18:57:12.499482 934 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 50c83e7e69974b90a46fe19124750a8d: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:12.502065 934 log.cc:824] T 00000000000000000000000000000000 P 50c83e7e69974b90a46fe19124750a8d: Log is configured to *not* fsync() on all Append() calls
I20260811 18:57:12.514758 934 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50c83e7e69974b90a46fe19124750a8d: No bootstrap required, opened a new log
I20260811 18:57:12.523440 940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:12.461043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43826 (local address 127.29.112.190:39833)
0811 18:57:12.464024 (+ 2981us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:12.464057 (+ 33us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:12.464148 (+ 91us) server_negotiation.cc:407] Connection header received
0811 18:57:12.468354 (+ 4206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:12.468399 (+ 45us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:12.469158 (+ 759us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:12.469759 (+ 601us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:12.474801 (+ 5042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:12.477423 (+ 2622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:12.481527 (+ 4104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:12.483347 (+ 1820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:12.483595 (+ 248us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:12.498451 (+ 14856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:12.498472 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:12.498509 (+ 37us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:12.498700 (+ 191us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:12.508828 (+ 10128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:12.511128 (+ 2300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:12.511164 (+ 36us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:12.511199 (+ 35us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:12.512435 (+ 1236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:12.513742 (+ 1307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:12.513762 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:12.513767 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:12.516472 (+ 2705us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:12.517355 (+ 883us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:12.518012 (+ 657us) server_negotiation.cc:299] Negotiation successful
0811 18:57:12.519405 (+ 1393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2458,"thread_start_us":1750,"threads_started":1}
I20260811 18:57:12.536278 934 raft_consensus.cc:348] T 00000000000000000000000000000000 P 50c83e7e69974b90a46fe19124750a8d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50c83e7e69974b90a46fe19124750a8d" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 39833 } }
I20260811 18:57:12.537436 934 raft_consensus.cc:374] T 00000000000000000000000000000000 P 50c83e7e69974b90a46fe19124750a8d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:12.537691 934 raft_consensus.cc:729] T 00000000000000000000000000000000 P 50c83e7e69974b90a46fe19124750a8d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50c83e7e69974b90a46fe19124750a8d, State: Initialized, Role: FOLLOWER
I20260811 18:57:12.539247 934 consensus_queue.cc:260] T 00000000000000000000000000000000 P 50c83e7e69974b90a46fe19124750a8d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50c83e7e69974b90a46fe19124750a8d" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 39833 } }
I20260811 18:57:12.539702 934 raft_consensus.cc:388] T 00000000000000000000000000000000 P 50c83e7e69974b90a46fe19124750a8d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:57:12.539877 934 raft_consensus.cc:482] T 00000000000000000000000000000000 P 50c83e7e69974b90a46fe19124750a8d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:57:12.540105 934 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 50c83e7e69974b90a46fe19124750a8d [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:12.544691 934 raft_consensus.cc:504] T 00000000000000000000000000000000 P 50c83e7e69974b90a46fe19124750a8d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50c83e7e69974b90a46fe19124750a8d" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 39833 } }
I20260811 18:57:12.545861 934 leader_election.cc:302] T 00000000000000000000000000000000 P 50c83e7e69974b90a46fe19124750a8d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 50c83e7e69974b90a46fe19124750a8d; no voters:
I20260811 18:57:12.547160 934 leader_election.cc:288] T 00000000000000000000000000000000 P 50c83e7e69974b90a46fe19124750a8d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:57:12.547390 946 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 50c83e7e69974b90a46fe19124750a8d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:12.548619 946 raft_consensus.cc:686] T 00000000000000000000000000000000 P 50c83e7e69974b90a46fe19124750a8d [term 1 LEADER]: Becoming Leader. State: Replica: 50c83e7e69974b90a46fe19124750a8d, State: Running, Role: LEADER
I20260811 18:57:12.549546 946 consensus_queue.cc:237] T 00000000000000000000000000000000 P 50c83e7e69974b90a46fe19124750a8d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50c83e7e69974b90a46fe19124750a8d" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 39833 } }
I20260811 18:57:12.550731 934 sys_catalog.cc:564] T 00000000000000000000000000000000 P 50c83e7e69974b90a46fe19124750a8d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:57:12.555595 948 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50c83e7e69974b90a46fe19124750a8d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 50c83e7e69974b90a46fe19124750a8d. Latest consensus state: current_term: 1 leader_uuid: "50c83e7e69974b90a46fe19124750a8d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50c83e7e69974b90a46fe19124750a8d" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 39833 } } }
I20260811 18:57:12.556761 948 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50c83e7e69974b90a46fe19124750a8d [sys.catalog]: This master's current role is: LEADER
I20260811 18:57:12.557087 947 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50c83e7e69974b90a46fe19124750a8d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "50c83e7e69974b90a46fe19124750a8d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50c83e7e69974b90a46fe19124750a8d" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 39833 } } }
I20260811 18:57:12.557621 947 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50c83e7e69974b90a46fe19124750a8d [sys.catalog]: This master's current role is: LEADER
I20260811 18:57:12.560537 955 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:57:12.572933 955 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:57:12.592878 955 catalog_manager.cc:1347] Generated new cluster ID: 6912e35dd42f4e67b3f0fa04c0febd3f
I20260811 18:57:12.593466 955 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:57:12.629612 955 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:57:12.632126 955 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:57:12.648088 955 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 50c83e7e69974b90a46fe19124750a8d: Generated new TSK 0
I20260811 18:57:12.650059 955 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.112.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.129.keytab.
Entry for principal kudu/127.29.112.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.129.keytab.
W20260811 18:57:12.774008 30146 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.29.112.129: real 0.101s user 0.003s sys 0.026s
I20260811 18:57:12.778378 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.129:0
--local_ip_for_outbound_sockets=127.29.112.129
--webserver_interface=127.29.112.129
--webserver_port=0
--tserver_master_addrs=127.29.112.190:39833
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.129.keytab
--principal=kudu/127.29.112.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:39255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:57:13.450656 975 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:57:13.451401 975 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:57:13.451764 975 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:57:13.471944 975 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:57:13.472231 975 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:57:13.472440 975 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 18:57:13.472581 975 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 18:57:13.472769 975 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.129
I20260811 18:57:13.496821 975 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:39255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.129.keytab
--principal=kudu/127.29.112.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.112.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.112.190:39833
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.129
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:57:13.499085 975 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:57:13.501219 975 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:57:13.526829 984 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:13.526937 987 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:13.526875 991 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:57:13.529102 975 server_base.cc:1034] running on GCE node
I20260811 18:57:13.530133 975 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:57:13.531780 975 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:57:13.533113 975 hybrid_clock.cc:648] HybridClock initialized: now 1786474633533025 us; error 104 us; skew 500 ppm
I20260811 18:57:13.533841 975 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:57:13 dist-test-slave-h6n0 krb5kdc[771](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474633, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:57:13.544488 975 init.cc:377] Logged in from keytab as kudu/127.29.112.129@KRBTEST.COM (short username kudu)
I20260811 18:57:13.547147 975 webserver.cc:460] Webserver started at http://127.29.112.129:40661/ using document root <none> and password file <none>
I20260811 18:57:13.548449 975 fs_manager.cc:359] Metadata directory not provided
I20260811 18:57:13.548619 975 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:57:13.549098 975 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:57:13.554627 975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/minicluster-data/ts-0/data/instance:
uuid: "937550b3337d417c9d5ec16caa6f3b09"
format_stamp: "Formatted at 2026-08-11 18:57:13 on dist-test-slave-h6n0"
server_key: "a57190afa405d90c10df30b25615a1f9"
server_key_iv: "694b859e0275672f212d8564a659e898"
server_key_version: "encryptionkey@0"
I20260811 18:57:13.555965 975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/minicluster-data/ts-0/wal/instance:
uuid: "937550b3337d417c9d5ec16caa6f3b09"
format_stamp: "Formatted at 2026-08-11 18:57:13 on dist-test-slave-h6n0"
server_key: "a57190afa405d90c10df30b25615a1f9"
server_key_iv: "694b859e0275672f212d8564a659e898"
server_key_version: "encryptionkey@0"
I20260811 18:57:13.567765 975 fs_manager.cc:691] Time spent creating directory manager: real 0.011s user 0.007s sys 0.004s
I20260811 18:57:13.575335 1003 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:13.577790 975 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.001s
I20260811 18:57:13.578013 975 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/minicluster-data/ts-0/wal
uuid: "937550b3337d417c9d5ec16caa6f3b09"
format_stamp: "Formatted at 2026-08-11 18:57:13 on dist-test-slave-h6n0"
server_key: "a57190afa405d90c10df30b25615a1f9"
server_key_iv: "694b859e0275672f212d8564a659e898"
server_key_version: "encryptionkey@0"
I20260811 18:57:13.578341 975 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:57:13.626672 975 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:57:13.635991 975 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:57:13.636507 975 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:57:13.641541 975 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:57:13.641698 975 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:13.641908 975 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:57:13.642107 975 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 18:57:13 dist-test-slave-h6n0 krb5kdc[771](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474633, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.129@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 18:57:13.697710 1024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:13.642246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.129:34415 (local address 127.29.112.190:39833)
0811 18:57:13.643821 (+ 1575us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:13.643840 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:13.645624 (+ 1784us) server_negotiation.cc:407] Connection header received
0811 18:57:13.652372 (+ 6748us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:13.652392 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:13.652586 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:13.652801 (+ 215us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:13.658777 (+ 5976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:13.660942 (+ 2165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:13.668225 (+ 7283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:13.669555 (+ 1330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:13.669783 (+ 228us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:13.680500 (+ 10717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:13.680515 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:13.680541 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:13.680651 (+ 110us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:13.687953 (+ 7302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:13.691742 (+ 3789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:13.691749 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:13.691752 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:13.692034 (+ 282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:13.693098 (+ 1064us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:13.693107 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:13.693124 (+ 17us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:13.693973 (+ 849us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:13.694315 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:13.695973 (+ 1658us) server_negotiation.cc:299] Negotiation successful
0811 18:57:13.696546 (+ 573us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1371,"thread_start_us":1115,"threads_started":1}
I20260811 18:57:13.699641 1025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:13.643592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:39833 (local address 127.29.112.129:34415)
0811 18:57:13.645256 (+ 1664us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:13.645366 (+ 110us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:13.651642 (+ 6276us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:13.654082 (+ 2440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:13.654119 (+ 37us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:13.656029 (+ 1910us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:57:13.658213 (+ 2184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:13.658282 (+ 69us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:13.661363 (+ 3081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:13.661385 (+ 22us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:13.667890 (+ 6505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:13.667938 (+ 48us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:13.670492 (+ 2554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:13.670551 (+ 59us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:13.671285 (+ 734us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:13.673278 (+ 1993us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:57:13.673423 (+ 145us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:57:13.679969 (+ 6546us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:57:13.688273 (+ 8304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:13.688296 (+ 23us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:13.688320 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:13.691470 (+ 3150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:13.692446 (+ 976us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:13.692455 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:13.692472 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:13.692844 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:13.694231 (+ 1387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:57:13.694251 (+ 20us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:57:13.695147 (+ 896us) client_negotiation.cc:769] Sending connection context
0811 18:57:13.695723 (+ 576us) client_negotiation.cc:240] Negotiation successful
0811 18:57:13.696617 (+ 894us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1091,"thread_start_us":672,"threads_started":1}
I20260811 18:57:13.732496 975 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.129:32865
I20260811 18:57:13.732576 1206 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.129:32865 every 8 connection(s)
I20260811 18:57:13.735911 975 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/minicluster-data/ts-0/data/info.pb
I20260811 18:57:13.741498 1207 heartbeater.cc:347] Connected to a master server at 127.29.112.190:39833
I20260811 18:57:13.741920 1207 heartbeater.cc:464] Registering TS with master...
I20260811 18:57:13.743156 1207 heartbeater.cc:511] Master 127.29.112.190:39833 requested a full tablet report, sending...
I20260811 18:57:13.745231 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 975
I20260811 18:57:13.745592 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/minicluster-data/ts-0/wal/instance
I20260811 18:57:13.747174 30146 external_mini_cluster.cc:1428] Setting key 8f5bba858e2ff3263af51a987c3f8bd3
I20260811 18:57:13.749539 853 ts_manager.cc:194] Registered new tserver with Master: 937550b3337d417c9d5ec16caa6f3b09 (127.29.112.129:32865)
I20260811 18:57:13.751349 1024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:13.715038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.129:34805 (local address 127.29.112.190:39833)
0811 18:57:13.715373 (+ 335us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:13.715394 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:13.718976 (+ 3582us) server_negotiation.cc:407] Connection header received
0811 18:57:13.719572 (+ 596us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:13.719584 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:13.719799 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:13.720014 (+ 215us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:13.723517 (+ 3503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:13.725768 (+ 2251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:13.731003 (+ 5235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:13.732345 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:13.732569 (+ 224us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:13.737329 (+ 4760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:13.737338 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:13.737359 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:13.737446 (+ 87us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:13.744525 (+ 7079us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:13.746396 (+ 1871us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:13.746405 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:13.746411 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:13.747038 (+ 627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:13.748472 (+ 1434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:13.748490 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:13.748494 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:13.749142 (+ 648us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:13.749434 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:13.750258 (+ 824us) server_negotiation.cc:299] Negotiation successful
0811 18:57:13.750745 (+ 487us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260811 18:57:13.751482 1165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:13.714885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:39833 (local address 127.29.112.129:34805)
0811 18:57:13.718751 (+ 3866us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:13.718792 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:13.719242 (+ 450us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:13.720153 (+ 911us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:13.720162 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:13.721828 (+ 1666us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:57:13.723065 (+ 1237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:13.723111 (+ 46us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:13.726167 (+ 3056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:13.726186 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:13.730624 (+ 4438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:13.730675 (+ 51us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:13.733122 (+ 2447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:13.733144 (+ 22us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:13.733468 (+ 324us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:13.734713 (+ 1245us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:57:13.734769 (+ 56us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:57:13.736864 (+ 2095us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:57:13.744980 (+ 8116us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:13.744987 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:13.744990 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:13.746155 (+ 1165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:13.747508 (+ 1353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:13.747518 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:13.747523 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:13.747979 (+ 456us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:13.749360 (+ 1381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:57:13.749378 (+ 18us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:57:13.749726 (+ 348us) client_negotiation.cc:769] Sending connection context
0811 18:57:13.750030 (+ 304us) client_negotiation.cc:240] Negotiation successful
0811 18:57:13.750654 (+ 624us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3310,"thread_start_us":2877,"threads_started":1}
I20260811 18:57:13.753042 853 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.129@KRBTEST.COM'} at 127.29.112.129:34415
WARNING: no policy specified for kudu/127.29.112.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.130.keytab.
Entry for principal kudu/127.29.112.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.130.keytab.
W20260811 18:57:14.269296 30146 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.29.112.130: real 0.516s user 0.000s sys 0.025s
I20260811 18:57:14.271026 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.130:0
--local_ip_for_outbound_sockets=127.29.112.130
--webserver_interface=127.29.112.130
--webserver_port=0
--tserver_master_addrs=127.29.112.190:39833
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.130.keytab
--principal=kudu/127.29.112.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:39255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 18:57:14.757815 1207 heartbeater.cc:503] Master 127.29.112.190:39833 was elected leader, sending a full tablet report...
W20260811 18:57:14.872548 1213 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:57:14.873142 1213 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:57:14.873461 1213 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:57:14.887863 1213 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:57:14.888156 1213 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:57:14.888409 1213 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 18:57:14.888566 1213 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 18:57:14.888803 1213 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.130
I20260811 18:57:14.909021 1213 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:39255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.130.keytab
--principal=kudu/127.29.112.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.112.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.112.190:39833
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.130
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:57:14.911518 1213 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:57:14.914167 1213 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:57:14.939522 1234 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:14.939796 1228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:14.940343 1227 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:57:14.942023 1213 server_base.cc:1034] running on GCE node
I20260811 18:57:14.943397 1213 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:57:14.944866 1213 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:57:14.946169 1213 hybrid_clock.cc:648] HybridClock initialized: now 1786474634946053 us; error 94 us; skew 500 ppm
I20260811 18:57:14.946695 1213 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:57:14 dist-test-slave-h6n0 krb5kdc[771](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474634, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:57:14.952498 1213 init.cc:377] Logged in from keytab as kudu/127.29.112.130@KRBTEST.COM (short username kudu)
I20260811 18:57:14.955021 1213 webserver.cc:460] Webserver started at http://127.29.112.130:44011/ using document root <none> and password file <none>
I20260811 18:57:14.956138 1213 fs_manager.cc:359] Metadata directory not provided
I20260811 18:57:14.956439 1213 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:57:14.957059 1213 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:57:14.961790 1213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/minicluster-data/ts-1/data/instance:
uuid: "482b0d44ef0941dd9f67913d024d29ba"
format_stamp: "Formatted at 2026-08-11 18:57:14 on dist-test-slave-h6n0"
server_key: "c62cc9253e05cf7350ba7fe492edca63"
server_key_iv: "479faf8ce6bf9460558aa3cfed833c07"
server_key_version: "encryptionkey@0"
I20260811 18:57:14.963146 1213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/minicluster-data/ts-1/wal/instance:
uuid: "482b0d44ef0941dd9f67913d024d29ba"
format_stamp: "Formatted at 2026-08-11 18:57:14 on dist-test-slave-h6n0"
server_key: "c62cc9253e05cf7350ba7fe492edca63"
server_key_iv: "479faf8ce6bf9460558aa3cfed833c07"
server_key_version: "encryptionkey@0"
I20260811 18:57:14.972921 1213 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.003s sys 0.005s
I20260811 18:57:14.978981 1246 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:14.981599 1213 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.001s
I20260811 18:57:14.981882 1213 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/minicluster-data/ts-1/wal
uuid: "482b0d44ef0941dd9f67913d024d29ba"
format_stamp: "Formatted at 2026-08-11 18:57:14 on dist-test-slave-h6n0"
server_key: "c62cc9253e05cf7350ba7fe492edca63"
server_key_iv: "479faf8ce6bf9460558aa3cfed833c07"
server_key_version: "encryptionkey@0"
I20260811 18:57:14.982256 1213 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:57:15.045054 1213 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:57:15.053849 1213 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:57:15.054335 1213 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:57:15.059541 1213 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:57:15.059777 1213 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:15.060002 1213 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:57:15.060179 1213 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 18:57:15 dist-test-slave-h6n0 krb5kdc[771](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474634, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.130@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 18:57:15.112133 1266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:15.060493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.130:58971 (local address 127.29.112.190:39833)
0811 18:57:15.061757 (+ 1264us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:15.061765 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:15.063685 (+ 1920us) server_negotiation.cc:407] Connection header received
0811 18:57:15.069877 (+ 6192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:15.069887 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:15.070086 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:15.070346 (+ 260us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:15.073906 (+ 3560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:15.075689 (+ 1783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:15.081289 (+ 5600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:15.083075 (+ 1786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:15.083397 (+ 322us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:15.099983 (+ 16586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:15.099999 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:15.100002 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:15.100070 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:15.105852 (+ 5782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:15.107362 (+ 1510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:15.107381 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:15.107384 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:15.107594 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:15.108385 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:15.108398 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:15.108400 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:15.108964 (+ 564us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:15.109470 (+ 506us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:15.110914 (+ 1444us) server_negotiation.cc:299] Negotiation successful
0811 18:57:15.111303 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1064,"thread_start_us":848,"threads_started":1}
I20260811 18:57:15.115114 1268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:15.061811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:39833 (local address 127.29.112.130:58971)
0811 18:57:15.063267 (+ 1456us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:15.063426 (+ 159us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:15.069083 (+ 5657us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:15.070824 (+ 1741us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:15.070846 (+ 22us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:15.072106 (+ 1260us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:57:15.073614 (+ 1508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:15.073635 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:15.075987 (+ 2352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:15.075996 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:15.080931 (+ 4935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:15.080962 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:15.083741 (+ 2779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:15.083764 (+ 23us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:15.084174 (+ 410us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:15.086063 (+ 1889us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:57:15.086173 (+ 110us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:57:15.099548 (+ 13375us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:57:15.106141 (+ 6593us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:15.106171 (+ 30us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:15.106207 (+ 36us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:15.107156 (+ 949us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:15.107794 (+ 638us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:15.107808 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:15.107814 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:15.108214 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:15.109397 (+ 1183us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:57:15.109422 (+ 25us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:57:15.110198 (+ 776us) client_negotiation.cc:769] Sending connection context
0811 18:57:15.110740 (+ 542us) client_negotiation.cc:240] Negotiation successful
0811 18:57:15.111833 (+ 1093us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":920,"thread_start_us":628,"threads_started":1}
I20260811 18:57:15.145998 1213 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.130:33619
I20260811 18:57:15.146137 1465 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.130:33619 every 8 connection(s)
I20260811 18:57:15.148972 1213 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/minicluster-data/ts-1/data/info.pb
I20260811 18:57:15.150622 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 1213
I20260811 18:57:15.150993 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/minicluster-data/ts-1/wal/instance
I20260811 18:57:15.152307 30146 external_mini_cluster.cc:1428] Setting key ec06e30f142fe5597a9055ceb8c7e049
I20260811 18:57:15.153417 1466 heartbeater.cc:347] Connected to a master server at 127.29.112.190:39833
I20260811 18:57:15.153937 1466 heartbeater.cc:464] Registering TS with master...
I20260811 18:57:15.156236 1466 heartbeater.cc:511] Master 127.29.112.190:39833 requested a full tablet report, sending...
I20260811 18:57:15.159680 852 ts_manager.cc:194] Registered new tserver with Master: 482b0d44ef0941dd9f67913d024d29ba (127.29.112.130:33619)
I20260811 18:57:15.162258 852 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.130@KRBTEST.COM'} at 127.29.112.130:58971
I20260811 18:57:15.165351 1434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:15.130756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:39833 (local address 127.29.112.130:54953)
0811 18:57:15.132244 (+ 1488us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:15.132283 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:15.132782 (+ 499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:15.133978 (+ 1196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:15.133985 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:15.136719 (+ 2734us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:57:15.137779 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:15.137842 (+ 63us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:15.140782 (+ 2940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:15.140791 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:15.143000 (+ 2209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:15.143024 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:15.144201 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:15.144208 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:15.144647 (+ 439us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:15.147185 (+ 2538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:57:15.147227 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:57:15.148968 (+ 1741us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:57:15.157697 (+ 8729us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:15.157717 (+ 20us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:15.157721 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:15.158936 (+ 1215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:15.159971 (+ 1035us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:15.159994 (+ 23us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:15.159999 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:15.160379 (+ 380us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:15.161924 (+ 1545us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:57:15.161932 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:57:15.163190 (+ 1258us) client_negotiation.cc:769] Sending connection context
0811 18:57:15.163521 (+ 331us) client_negotiation.cc:240] Negotiation successful
0811 18:57:15.164317 (+ 796us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1170,"thread_start_us":911,"threads_started":1}
I20260811 18:57:15.167869 1266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:15.130064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.130:54953 (local address 127.29.112.190:39833)
0811 18:57:15.130600 (+ 536us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:15.130612 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:15.132524 (+ 1912us) server_negotiation.cc:407] Connection header received
0811 18:57:15.133149 (+ 625us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:15.133159 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:15.133405 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:15.133577 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:15.138412 (+ 4835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:15.140415 (+ 2003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:15.143217 (+ 2802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:15.143887 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:15.144071 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:15.149308 (+ 5237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:15.149324 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:15.149329 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:15.149398 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:15.157382 (+ 7984us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:15.159332 (+ 1950us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:15.159338 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:15.159355 (+ 17us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:15.159680 (+ 325us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:15.160746 (+ 1066us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:15.160766 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:15.160770 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:15.161651 (+ 881us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:15.166139 (+ 4488us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:15.166421 (+ 282us) server_negotiation.cc:299] Negotiation successful
0811 18:57:15.166944 (+ 523us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":15,"server-negotiator.queue_time_us":149}
WARNING: no policy specified for kudu/127.29.112.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.131.keytab.
Entry for principal kudu/127.29.112.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.131.keytab.
W20260811 18:57:15.263679 30146 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.29.112.131: real 0.104s user 0.003s sys 0.023s
I20260811 18:57:15.264973 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.131:0
--local_ip_for_outbound_sockets=127.29.112.131
--webserver_interface=127.29.112.131
--webserver_port=0
--tserver_master_addrs=127.29.112.190:39833
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.131.keytab
--principal=kudu/127.29.112.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:39255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:57:15.647571 1474 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:57:15.648002 1474 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:57:15.648192 1474 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:57:15.657387 1474 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:57:15.657531 1474 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:57:15.657683 1474 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 18:57:15.657785 1474 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 18:57:15.657917 1474 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.131
I20260811 18:57:15.669065 1474 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:39255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.131.keytab
--principal=kudu/127.29.112.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.112.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.112.190:39833
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.131
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:57:15.670437 1474 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:57:15.671815 1474 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:57:15.686414 1488 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:15.686410 1493 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:15.686349 1490 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:57:15.692230 1474 server_base.cc:1034] running on GCE node
I20260811 18:57:15.692960 1474 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:57:15.694095 1474 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:57:15.695323 1474 hybrid_clock.cc:648] HybridClock initialized: now 1786474635695233 us; error 80 us; skew 500 ppm
I20260811 18:57:15.695681 1474 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:57:15 dist-test-slave-h6n0 krb5kdc[771](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474635, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:57:15.699626 1474 init.cc:377] Logged in from keytab as kudu/127.29.112.131@KRBTEST.COM (short username kudu)
I20260811 18:57:15.701821 1474 webserver.cc:460] Webserver started at http://127.29.112.131:36235/ using document root <none> and password file <none>
I20260811 18:57:15.702700 1474 fs_manager.cc:359] Metadata directory not provided
I20260811 18:57:15.702860 1474 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:57:15.703195 1474 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:57:15.706379 1474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/minicluster-data/ts-2/data/instance:
uuid: "4c0fb9abd83245b289dc12805882cc9e"
format_stamp: "Formatted at 2026-08-11 18:57:15 on dist-test-slave-h6n0"
server_key: "4447de774f5a48a12d0b5f23353d6572"
server_key_iv: "cc75d1166a3935e89cdc2c72056f4f02"
server_key_version: "encryptionkey@0"
I20260811 18:57:15.707224 1474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/minicluster-data/ts-2/wal/instance:
uuid: "4c0fb9abd83245b289dc12805882cc9e"
format_stamp: "Formatted at 2026-08-11 18:57:15 on dist-test-slave-h6n0"
server_key: "4447de774f5a48a12d0b5f23353d6572"
server_key_iv: "cc75d1166a3935e89cdc2c72056f4f02"
server_key_version: "encryptionkey@0"
I20260811 18:57:15.712832 1474 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260811 18:57:15.716616 1504 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:15.718242 1474 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 18:57:15.718463 1474 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/minicluster-data/ts-2/wal
uuid: "4c0fb9abd83245b289dc12805882cc9e"
format_stamp: "Formatted at 2026-08-11 18:57:15 on dist-test-slave-h6n0"
server_key: "4447de774f5a48a12d0b5f23353d6572"
server_key_iv: "cc75d1166a3935e89cdc2c72056f4f02"
server_key_version: "encryptionkey@0"
I20260811 18:57:15.718757 1474 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:57:15.732986 1474 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:57:15.737423 1474 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:57:15.737709 1474 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:57:15.740458 1474 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:57:15.740589 1474 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:15.740736 1474 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:57:15.740844 1474 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 18:57:15 dist-test-slave-h6n0 krb5kdc[771](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474635, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.131@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 18:57:15.767261 1517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:15.740839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.131:58177 (local address 127.29.112.190:39833)
0811 18:57:15.741423 (+ 584us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:15.741430 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:15.742713 (+ 1283us) server_negotiation.cc:407] Connection header received
0811 18:57:15.744758 (+ 2045us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:15.744763 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:15.744861 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:15.744990 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:15.747300 (+ 2310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:15.748665 (+ 1365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:15.751263 (+ 2598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:15.751961 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:15.752110 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:15.758206 (+ 6096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:15.758213 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:15.758217 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:15.758255 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:15.762517 (+ 4262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:15.764364 (+ 1847us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:15.764370 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:15.764373 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:15.764528 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:15.765085 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:15.765090 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:15.765095 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:15.765443 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:15.765618 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:15.766585 (+ 967us) server_negotiation.cc:299] Negotiation successful
0811 18:57:15.766819 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":246,"threads_started":1}
I20260811 18:57:15.768352 1520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:15.741472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:39833 (local address 127.29.112.131:58177)
0811 18:57:15.742513 (+ 1041us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:15.742595 (+ 82us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:15.744485 (+ 1890us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:15.745228 (+ 743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:15.745235 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:15.745996 (+ 761us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:57:15.747015 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:15.747049 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:15.748854 (+ 1805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:15.748865 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:15.751108 (+ 2243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:15.751120 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:15.752228 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:15.752236 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:15.752753 (+ 517us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:15.753782 (+ 1029us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:57:15.753820 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:57:15.757891 (+ 4071us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:57:15.763276 (+ 5385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:15.763290 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:15.763315 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:15.764193 (+ 878us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:15.764653 (+ 460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:15.764659 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:15.764662 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:15.764951 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:15.765586 (+ 635us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:57:15.765596 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:57:15.766096 (+ 500us) client_negotiation.cc:769] Sending connection context
0811 18:57:15.766429 (+ 333us) client_negotiation.cc:240] Negotiation successful
0811 18:57:15.767112 (+ 683us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":753,"thread_start_us":471,"threads_started":1}
I20260811 18:57:15.789604 1517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:15.774076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.131:56547 (local address 127.29.112.190:39833)
0811 18:57:15.774256 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:15.774263 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:15.775151 (+ 888us) server_negotiation.cc:407] Connection header received
0811 18:57:15.775355 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:15.775360 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:15.775430 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:15.775539 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:15.776945 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:15.777970 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:15.779928 (+ 1958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:15.780596 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:15.780737 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:15.783421 (+ 2684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:15.783426 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:15.783429 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:15.783458 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:15.786230 (+ 2772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:15.787345 (+ 1115us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:15.787353 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:15.787358 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:15.787529 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:15.788030 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:15.788038 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:15.788043 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:15.788450 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:15.788612 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:15.789045 (+ 433us) server_negotiation.cc:299] Negotiation successful
0811 18:57:15.789264 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 18:57:15.789708 1613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:15.774223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:39833 (local address 127.29.112.131:56547)
0811 18:57:15.775048 (+ 825us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:15.775064 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:15.775222 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:15.775569 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:15.775574 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:15.776276 (+ 702us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:57:15.776737 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:15.776756 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:15.778162 (+ 1406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:15.778166 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:15.779697 (+ 1531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:15.779719 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:15.780788 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:15.780794 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:15.780961 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:15.781712 (+ 751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:57:15.781745 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:57:15.783221 (+ 1476us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:57:15.786426 (+ 3205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:15.786435 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:15.786442 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:15.787180 (+ 738us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:15.787660 (+ 480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:15.787665 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:15.787668 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:15.787817 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:15.788579 (+ 762us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:57:15.788584 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:57:15.788768 (+ 184us) client_negotiation.cc:769] Sending connection context
0811 18:57:15.788912 (+ 144us) client_negotiation.cc:240] Negotiation successful
0811 18:57:15.789235 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":680,"thread_start_us":478,"threads_started":1}
I20260811 18:57:15.805480 1474 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.131:45211
I20260811 18:57:15.805536 1713 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.131:45211 every 8 connection(s)
I20260811 18:57:15.807107 1474 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/minicluster-data/ts-2/data/info.pb
I20260811 18:57:15.810389 1716 heartbeater.cc:347] Connected to a master server at 127.29.112.190:39833
I20260811 18:57:15.810767 1716 heartbeater.cc:464] Registering TS with master...
I20260811 18:57:15.811511 1716 heartbeater.cc:511] Master 127.29.112.190:39833 requested a full tablet report, sending...
I20260811 18:57:15.813382 852 ts_manager.cc:194] Registered new tserver with Master: 4c0fb9abd83245b289dc12805882cc9e (127.29.112.131:45211)
I20260811 18:57:15.814513 852 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.131@KRBTEST.COM'} at 127.29.112.131:58177
I20260811 18:57:15.817282 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 1474
I20260811 18:57:15.817467 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/minicluster-data/ts-2/wal/instance
I20260811 18:57:15.817864 30146 external_mini_cluster.cc:1428] Setting key 6e6df45d6570628b072175091f174f58
I20260811 18:57:15.821287 30146 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 18:57:15 dist-test-slave-h6n0 krb5kdc[771](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474635, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 18:57:15 dist-test-slave-h6n0 krb5kdc[771](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474635, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 18:57:15.877125 1517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:15.861897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43842 (local address 127.29.112.190:39833)
0811 18:57:15.862152 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:15.862161 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:15.862380 (+ 219us) server_negotiation.cc:407] Connection header received
0811 18:57:15.862651 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:15.862656 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:15.862739 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:15.862854 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:15.864356 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:15.865131 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:15.866431 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:15.866936 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:15.867126 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:15.871575 (+ 4449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:15.871580 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:15.871584 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:15.871618 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:15.874245 (+ 2627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:15.875094 (+ 849us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:15.875100 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:15.875104 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:15.875256 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:15.875664 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:15.875668 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:15.875670 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:15.875964 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:15.876252 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:15.876539 (+ 287us) server_negotiation.cc:299] Negotiation successful
0811 18:57:15.876769 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 18:57:15.881455 852 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43842:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:15.888516 852 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:15.923774 1739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:15.901814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.130:33619 (local address 127.0.0.1:38916)
0811 18:57:15.913456 (+ 11642us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:15.913480 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:15.913646 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:15.914656 (+ 1010us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:15.914666 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:15.914706 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:57:15.915201 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:15.915219 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:15.917325 (+ 2106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:15.917333 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:15.920484 (+ 3151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:15.920505 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:15.922299 (+ 1794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:15.922320 (+ 21us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:15.922635 (+ 315us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:15.922695 (+ 60us) client_negotiation.cc:769] Sending connection context
0811 18:57:15.922829 (+ 134us) client_negotiation.cc:240] Negotiation successful
0811 18:57:15.923211 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11451,"mutex_wait_us":1,"thread_start_us":11291,"threads_started":1}
I20260811 18:57:15.924484 1741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:15.902035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38916 (local address 127.29.112.130:33619)
0811 18:57:15.912666 (+ 10631us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:15.912680 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:15.913796 (+ 1116us) server_negotiation.cc:407] Connection header received
0811 18:57:15.913914 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:15.913924 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:15.914197 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:15.914395 (+ 198us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:57:15.915434 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:15.917172 (+ 1738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:15.920659 (+ 3487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:15.922019 (+ 1360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:15.922194 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:15.922387 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:15.922920 (+ 533us) server_negotiation.cc:299] Negotiation successful
0811 18:57:15.924209 (+ 1289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10369,"thread_start_us":10632,"threads_started":1}
I20260811 18:57:15.926518 1742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:15.901517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.131:45211 (local address 127.0.0.1:47630)
0811 18:57:15.911798 (+ 10281us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:15.911865 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:15.912045 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:15.918562 (+ 6517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:15.918569 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:15.918603 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:57:15.919126 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:15.919142 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:15.921606 (+ 2464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:15.921611 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:15.924032 (+ 2421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:15.924049 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:15.925684 (+ 1635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:15.925690 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:15.925859 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:15.925910 (+ 51us) client_negotiation.cc:769] Sending connection context
0811 18:57:15.926010 (+ 100us) client_negotiation.cc:240] Negotiation successful
0811 18:57:15.926142 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8894,"mutex_wait_us":54,"thread_start_us":15535,"threads_started":1}
I20260811 18:57:15.926590 1740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:15.901812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47630 (local address 127.29.112.131:45211)
0811 18:57:15.917869 (+ 16057us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:15.917878 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:15.917911 (+ 33us) server_negotiation.cc:407] Connection header received
0811 18:57:15.918006 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:15.918013 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:15.918270 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:15.919331 (+ 1061us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:57:15.919671 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:15.921439 (+ 1768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:15.924220 (+ 2781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:15.925458 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:15.925674 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:15.925836 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:15.926069 (+ 233us) server_negotiation.cc:299] Negotiation successful
0811 18:57:15.926224 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15890,"thread_start_us":15684,"threads_started":1}
I20260811 18:57:15.928109 1738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:15.905105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:32865 (local address 127.0.0.1:57678)
0811 18:57:15.916869 (+ 11764us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:15.916886 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:15.916985 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:15.919086 (+ 2101us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:15.919096 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:15.919121 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:57:15.919499 (+ 378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:15.919508 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:15.921206 (+ 1698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:15.921212 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:15.923493 (+ 2281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:15.923512 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:15.927265 (+ 3753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:15.927273 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:15.927437 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:15.927469 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 18:57:15.927569 (+ 100us) client_negotiation.cc:240] Negotiation successful
0811 18:57:15.927701 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11649,"mutex_wait_us":60,"thread_start_us":4983,"threads_started":1}
I20260811 18:57:15.928359 1745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:15.905646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57678 (local address 127.29.112.129:32865)
0811 18:57:15.918260 (+ 12614us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:15.918269 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:15.918297 (+ 28us) server_negotiation.cc:407] Connection header received
0811 18:57:15.918402 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:15.918410 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:15.918641 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:15.918837 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:57:15.919724 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:15.921033 (+ 1309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:15.923715 (+ 2682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:15.925518 (+ 1803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:15.927243 (+ 1725us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:15.927426 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:15.927630 (+ 204us) server_negotiation.cc:299] Negotiation successful
0811 18:57:15.927804 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12456,"thread_start_us":12293,"threads_started":1}
I20260811 18:57:15.928578 1344 tablet_service.cc:1467] Processing CreateTablet for tablet be0213a1e743498d9455d9807a69f0b2 (DEFAULT_TABLE table=test-table [id=32ca67adae6f4758a8e46eadc36bc92c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:15.930431 1586 tablet_service.cc:1467] Processing CreateTablet for tablet be0213a1e743498d9455d9807a69f0b2 (DEFAULT_TABLE table=test-table [id=32ca67adae6f4758a8e46eadc36bc92c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:15.930972 1344 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be0213a1e743498d9455d9807a69f0b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:15.931783 1586 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be0213a1e743498d9455d9807a69f0b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:15.932813 1096 tablet_service.cc:1467] Processing CreateTablet for tablet be0213a1e743498d9455d9807a69f0b2 (DEFAULT_TABLE table=test-table [id=32ca67adae6f4758a8e46eadc36bc92c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:15.934202 1096 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be0213a1e743498d9455d9807a69f0b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:15.943735 1746 tablet_bootstrap.cc:492] T be0213a1e743498d9455d9807a69f0b2 P 4c0fb9abd83245b289dc12805882cc9e: Bootstrap starting.
I20260811 18:57:15.947100 1747 tablet_bootstrap.cc:492] T be0213a1e743498d9455d9807a69f0b2 P 482b0d44ef0941dd9f67913d024d29ba: Bootstrap starting.
I20260811 18:57:15.948385 1746 tablet_bootstrap.cc:654] T be0213a1e743498d9455d9807a69f0b2 P 4c0fb9abd83245b289dc12805882cc9e: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:15.949599 1748 tablet_bootstrap.cc:492] T be0213a1e743498d9455d9807a69f0b2 P 937550b3337d417c9d5ec16caa6f3b09: Bootstrap starting.
I20260811 18:57:15.949801 1746 log.cc:824] T be0213a1e743498d9455d9807a69f0b2 P 4c0fb9abd83245b289dc12805882cc9e: Log is configured to *not* fsync() on all Append() calls
I20260811 18:57:15.950868 1747 tablet_bootstrap.cc:654] T be0213a1e743498d9455d9807a69f0b2 P 482b0d44ef0941dd9f67913d024d29ba: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:15.952145 1747 log.cc:824] T be0213a1e743498d9455d9807a69f0b2 P 482b0d44ef0941dd9f67913d024d29ba: Log is configured to *not* fsync() on all Append() calls
I20260811 18:57:15.953029 1748 tablet_bootstrap.cc:654] T be0213a1e743498d9455d9807a69f0b2 P 937550b3337d417c9d5ec16caa6f3b09: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:15.953439 1746 tablet_bootstrap.cc:492] T be0213a1e743498d9455d9807a69f0b2 P 4c0fb9abd83245b289dc12805882cc9e: No bootstrap required, opened a new log
I20260811 18:57:15.953725 1746 ts_tablet_manager.cc:1397] T be0213a1e743498d9455d9807a69f0b2 P 4c0fb9abd83245b289dc12805882cc9e: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.004s
I20260811 18:57:15.954334 1748 log.cc:824] T be0213a1e743498d9455d9807a69f0b2 P 937550b3337d417c9d5ec16caa6f3b09: Log is configured to *not* fsync() on all Append() calls
I20260811 18:57:15.955336 1747 tablet_bootstrap.cc:492] T be0213a1e743498d9455d9807a69f0b2 P 482b0d44ef0941dd9f67913d024d29ba: No bootstrap required, opened a new log
I20260811 18:57:15.955749 1747 ts_tablet_manager.cc:1397] T be0213a1e743498d9455d9807a69f0b2 P 482b0d44ef0941dd9f67913d024d29ba: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260811 18:57:15.958629 1748 tablet_bootstrap.cc:492] T be0213a1e743498d9455d9807a69f0b2 P 937550b3337d417c9d5ec16caa6f3b09: No bootstrap required, opened a new log
I20260811 18:57:15.959074 1748 ts_tablet_manager.cc:1397] T be0213a1e743498d9455d9807a69f0b2 P 937550b3337d417c9d5ec16caa6f3b09: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.005s
I20260811 18:57:15.963670 1746 raft_consensus.cc:348] T be0213a1e743498d9455d9807a69f0b2 P 4c0fb9abd83245b289dc12805882cc9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c0fb9abd83245b289dc12805882cc9e" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45211 } } peers { permanent_uuid: "482b0d44ef0941dd9f67913d024d29ba" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33619 } } peers { permanent_uuid: "937550b3337d417c9d5ec16caa6f3b09" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 32865 } }
I20260811 18:57:15.964090 1746 raft_consensus.cc:374] T be0213a1e743498d9455d9807a69f0b2 P 4c0fb9abd83245b289dc12805882cc9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:15.964210 1746 raft_consensus.cc:729] T be0213a1e743498d9455d9807a69f0b2 P 4c0fb9abd83245b289dc12805882cc9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c0fb9abd83245b289dc12805882cc9e, State: Initialized, Role: FOLLOWER
I20260811 18:57:15.964888 1746 consensus_queue.cc:260] T be0213a1e743498d9455d9807a69f0b2 P 4c0fb9abd83245b289dc12805882cc9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c0fb9abd83245b289dc12805882cc9e" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45211 } } peers { permanent_uuid: "482b0d44ef0941dd9f67913d024d29ba" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33619 } } peers { permanent_uuid: "937550b3337d417c9d5ec16caa6f3b09" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 32865 } }
I20260811 18:57:15.966238 1716 heartbeater.cc:503] Master 127.29.112.190:39833 was elected leader, sending a full tablet report...
I20260811 18:57:15.967000 1746 ts_tablet_manager.cc:1428] T be0213a1e743498d9455d9807a69f0b2 P 4c0fb9abd83245b289dc12805882cc9e: Time spent starting tablet: real 0.013s user 0.009s sys 0.004s
I20260811 18:57:15.969125 1747 raft_consensus.cc:348] T be0213a1e743498d9455d9807a69f0b2 P 482b0d44ef0941dd9f67913d024d29ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c0fb9abd83245b289dc12805882cc9e" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45211 } } peers { permanent_uuid: "482b0d44ef0941dd9f67913d024d29ba" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33619 } } peers { permanent_uuid: "937550b3337d417c9d5ec16caa6f3b09" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 32865 } }
I20260811 18:57:15.969575 1747 raft_consensus.cc:374] T be0213a1e743498d9455d9807a69f0b2 P 482b0d44ef0941dd9f67913d024d29ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:15.969663 1747 raft_consensus.cc:729] T be0213a1e743498d9455d9807a69f0b2 P 482b0d44ef0941dd9f67913d024d29ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482b0d44ef0941dd9f67913d024d29ba, State: Initialized, Role: FOLLOWER
I20260811 18:57:15.970362 1747 consensus_queue.cc:260] T be0213a1e743498d9455d9807a69f0b2 P 482b0d44ef0941dd9f67913d024d29ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c0fb9abd83245b289dc12805882cc9e" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45211 } } peers { permanent_uuid: "482b0d44ef0941dd9f67913d024d29ba" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33619 } } peers { permanent_uuid: "937550b3337d417c9d5ec16caa6f3b09" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 32865 } }
I20260811 18:57:15.971626 1748 raft_consensus.cc:348] T be0213a1e743498d9455d9807a69f0b2 P 937550b3337d417c9d5ec16caa6f3b09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c0fb9abd83245b289dc12805882cc9e" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45211 } } peers { permanent_uuid: "482b0d44ef0941dd9f67913d024d29ba" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33619 } } peers { permanent_uuid: "937550b3337d417c9d5ec16caa6f3b09" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 32865 } }
I20260811 18:57:15.972040 1748 raft_consensus.cc:374] T be0213a1e743498d9455d9807a69f0b2 P 937550b3337d417c9d5ec16caa6f3b09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:15.972138 1748 raft_consensus.cc:729] T be0213a1e743498d9455d9807a69f0b2 P 937550b3337d417c9d5ec16caa6f3b09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 937550b3337d417c9d5ec16caa6f3b09, State: Initialized, Role: FOLLOWER
I20260811 18:57:15.972566 1466 heartbeater.cc:503] Master 127.29.112.190:39833 was elected leader, sending a full tablet report...
I20260811 18:57:15.972697 1747 ts_tablet_manager.cc:1428] T be0213a1e743498d9455d9807a69f0b2 P 482b0d44ef0941dd9f67913d024d29ba: Time spent starting tablet: real 0.017s user 0.016s sys 0.000s
I20260811 18:57:15.972862 1748 consensus_queue.cc:260] T be0213a1e743498d9455d9807a69f0b2 P 937550b3337d417c9d5ec16caa6f3b09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c0fb9abd83245b289dc12805882cc9e" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45211 } } peers { permanent_uuid: "482b0d44ef0941dd9f67913d024d29ba" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33619 } } peers { permanent_uuid: "937550b3337d417c9d5ec16caa6f3b09" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 32865 } }
I20260811 18:57:15.975857 1748 ts_tablet_manager.cc:1428] T be0213a1e743498d9455d9807a69f0b2 P 937550b3337d417c9d5ec16caa6f3b09: Time spent starting tablet: real 0.017s user 0.012s sys 0.002s
W20260811 18:57:15.992215 1209 tablet.cc:2369] T be0213a1e743498d9455d9807a69f0b2 P 937550b3337d417c9d5ec16caa6f3b09: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:57:16.016980 1756 raft_consensus.cc:482] T be0213a1e743498d9455d9807a69f0b2 P 482b0d44ef0941dd9f67913d024d29ba [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:16.017666 1756 raft_consensus.cc:504] T be0213a1e743498d9455d9807a69f0b2 P 482b0d44ef0941dd9f67913d024d29ba [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c0fb9abd83245b289dc12805882cc9e" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45211 } } peers { permanent_uuid: "482b0d44ef0941dd9f67913d024d29ba" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33619 } } peers { permanent_uuid: "937550b3337d417c9d5ec16caa6f3b09" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 32865 } }
I20260811 18:57:16.020205 1756 leader_election.cc:288] T be0213a1e743498d9455d9807a69f0b2 P 482b0d44ef0941dd9f67913d024d29ba [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4c0fb9abd83245b289dc12805882cc9e (127.29.112.131:45211), 937550b3337d417c9d5ec16caa6f3b09 (127.29.112.129:32865)
I20260811 18:57:16.028893 1740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:16.020856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.130:52853 (local address 127.29.112.131:45211)
0811 18:57:16.021086 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:16.021090 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:16.023395 (+ 2305us) server_negotiation.cc:407] Connection header received
0811 18:57:16.023609 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:16.023615 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:16.023693 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:16.023805 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:57:16.024537 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:16.025269 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:16.027128 (+ 1859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:16.027885 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:16.028095 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:16.028171 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:16.028485 (+ 314us) server_negotiation.cc:299] Negotiation successful
0811 18:57:16.028608 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 18:57:16.029072 1745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:16.021000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.130:44295 (local address 127.29.112.129:32865)
0811 18:57:16.021346 (+ 346us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:16.021352 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:16.023235 (+ 1883us) server_negotiation.cc:407] Connection header received
0811 18:57:16.023568 (+ 333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:16.023572 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:16.023642 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:16.023795 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:57:16.024770 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:16.025514 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:16.027356 (+ 1842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:16.028135 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:16.028316 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:16.028394 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:16.028685 (+ 291us) server_negotiation.cc:299] Negotiation successful
0811 18:57:16.028809 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177}
I20260811 18:57:16.029212 1760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:16.021562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:32865 (local address 127.29.112.130:44295)
0811 18:57:16.023080 (+ 1518us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:16.023106 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:16.023385 (+ 279us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:16.023821 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:16.023826 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:16.023900 (+ 74us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:57:16.024531 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:16.024556 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:16.025675 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:16.025680 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:16.027189 (+ 1509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:16.027200 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:16.028360 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:16.028366 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:16.028523 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:16.028545 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 18:57:16.028653 (+ 108us) client_negotiation.cc:240] Negotiation successful
0811 18:57:16.028787 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1371,"mutex_wait_us":3,"spinlock_wait_cycles":71936,"thread_start_us":988,"threads_started":1}
I20260811 18:57:16.029611 1759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:16.021557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.131:45211 (local address 127.29.112.130:52853)
0811 18:57:16.023229 (+ 1672us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:16.023255 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:16.023427 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:16.023831 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:16.023835 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:16.023890 (+ 55us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:57:16.024364 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:16.024375 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:16.025410 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:16.025416 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:16.026963 (+ 1547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:16.026976 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:16.028118 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:16.028125 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:16.028307 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:16.028341 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 18:57:16.028454 (+ 113us) client_negotiation.cc:240] Negotiation successful
0811 18:57:16.028594 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1500,"mutex_wait_us":30,"spinlock_wait_cycles":28544,"thread_start_us":859,"threads_started":1}
I20260811 18:57:16.030249 1126 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be0213a1e743498d9455d9807a69f0b2" candidate_uuid: "482b0d44ef0941dd9f67913d024d29ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "937550b3337d417c9d5ec16caa6f3b09" is_pre_election: true
I20260811 18:57:16.030537 1636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be0213a1e743498d9455d9807a69f0b2" candidate_uuid: "482b0d44ef0941dd9f67913d024d29ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c0fb9abd83245b289dc12805882cc9e" is_pre_election: true
I20260811 18:57:16.030850 1126 raft_consensus.cc:2454] T be0213a1e743498d9455d9807a69f0b2 P 937550b3337d417c9d5ec16caa6f3b09 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 482b0d44ef0941dd9f67913d024d29ba in term 0.
I20260811 18:57:16.031111 1636 raft_consensus.cc:2454] T be0213a1e743498d9455d9807a69f0b2 P 4c0fb9abd83245b289dc12805882cc9e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 482b0d44ef0941dd9f67913d024d29ba in term 0.
I20260811 18:57:16.031599 1258 leader_election.cc:302] T be0213a1e743498d9455d9807a69f0b2 P 482b0d44ef0941dd9f67913d024d29ba [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 482b0d44ef0941dd9f67913d024d29ba, 937550b3337d417c9d5ec16caa6f3b09; no voters:
I20260811 18:57:16.032043 1756 raft_consensus.cc:2781] T be0213a1e743498d9455d9807a69f0b2 P 482b0d44ef0941dd9f67913d024d29ba [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:16.032200 1756 raft_consensus.cc:482] T be0213a1e743498d9455d9807a69f0b2 P 482b0d44ef0941dd9f67913d024d29ba [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:16.032308 1756 raft_consensus.cc:3037] T be0213a1e743498d9455d9807a69f0b2 P 482b0d44ef0941dd9f67913d024d29ba [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:16.034688 1756 raft_consensus.cc:504] T be0213a1e743498d9455d9807a69f0b2 P 482b0d44ef0941dd9f67913d024d29ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c0fb9abd83245b289dc12805882cc9e" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45211 } } peers { permanent_uuid: "482b0d44ef0941dd9f67913d024d29ba" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33619 } } peers { permanent_uuid: "937550b3337d417c9d5ec16caa6f3b09" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 32865 } }
I20260811 18:57:16.035322 1756 leader_election.cc:288] T be0213a1e743498d9455d9807a69f0b2 P 482b0d44ef0941dd9f67913d024d29ba [CANDIDATE]: Term 1 election: Requested vote from peers 4c0fb9abd83245b289dc12805882cc9e (127.29.112.131:45211), 937550b3337d417c9d5ec16caa6f3b09 (127.29.112.129:32865)
I20260811 18:57:16.035728 1636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be0213a1e743498d9455d9807a69f0b2" candidate_uuid: "482b0d44ef0941dd9f67913d024d29ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c0fb9abd83245b289dc12805882cc9e"
I20260811 18:57:16.035761 1126 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be0213a1e743498d9455d9807a69f0b2" candidate_uuid: "482b0d44ef0941dd9f67913d024d29ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "937550b3337d417c9d5ec16caa6f3b09"
I20260811 18:57:16.035943 1636 raft_consensus.cc:3037] T be0213a1e743498d9455d9807a69f0b2 P 4c0fb9abd83245b289dc12805882cc9e [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:16.035962 1126 raft_consensus.cc:3037] T be0213a1e743498d9455d9807a69f0b2 P 937550b3337d417c9d5ec16caa6f3b09 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:16.038065 1126 raft_consensus.cc:2454] T be0213a1e743498d9455d9807a69f0b2 P 937550b3337d417c9d5ec16caa6f3b09 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 482b0d44ef0941dd9f67913d024d29ba in term 1.
I20260811 18:57:16.038179 1636 raft_consensus.cc:2454] T be0213a1e743498d9455d9807a69f0b2 P 4c0fb9abd83245b289dc12805882cc9e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 482b0d44ef0941dd9f67913d024d29ba in term 1.
I20260811 18:57:16.038558 1258 leader_election.cc:302] T be0213a1e743498d9455d9807a69f0b2 P 482b0d44ef0941dd9f67913d024d29ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 482b0d44ef0941dd9f67913d024d29ba, 937550b3337d417c9d5ec16caa6f3b09; no voters:
I20260811 18:57:16.038852 1756 raft_consensus.cc:2781] T be0213a1e743498d9455d9807a69f0b2 P 482b0d44ef0941dd9f67913d024d29ba [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:16.039274 1756 raft_consensus.cc:686] T be0213a1e743498d9455d9807a69f0b2 P 482b0d44ef0941dd9f67913d024d29ba [term 1 LEADER]: Becoming Leader. State: Replica: 482b0d44ef0941dd9f67913d024d29ba, State: Running, Role: LEADER
I20260811 18:57:16.039724 1756 consensus_queue.cc:237] T be0213a1e743498d9455d9807a69f0b2 P 482b0d44ef0941dd9f67913d024d29ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c0fb9abd83245b289dc12805882cc9e" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45211 } } peers { permanent_uuid: "482b0d44ef0941dd9f67913d024d29ba" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33619 } } peers { permanent_uuid: "937550b3337d417c9d5ec16caa6f3b09" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 32865 } }
I20260811 18:57:16.044617 852 catalog_manager.cc:5495] T be0213a1e743498d9455d9807a69f0b2 P 482b0d44ef0941dd9f67913d024d29ba reported cstate change: term changed from 0 to 1, leader changed from <none> to 482b0d44ef0941dd9f67913d024d29ba (127.29.112.130). New cstate: current_term: 1 leader_uuid: "482b0d44ef0941dd9f67913d024d29ba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c0fb9abd83245b289dc12805882cc9e" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45211 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "482b0d44ef0941dd9f67913d024d29ba" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "937550b3337d417c9d5ec16caa6f3b09" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 32865 } health_report { overall_health: UNKNOWN } } }
W20260811 18:57:16.059499 1718 tablet.cc:2369] T be0213a1e743498d9455d9807a69f0b2 P 4c0fb9abd83245b289dc12805882cc9e: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 11 18:57:16 dist-test-slave-h6n0 krb5kdc[771](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474632, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.190@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 18:57:16.119153 1517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:16.105390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43844 (local address 127.29.112.190:39833)
0811 18:57:16.105591 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:16.105596 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:16.106116 (+ 520us) server_negotiation.cc:407] Connection header received
0811 18:57:16.106320 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:16.106328 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:16.106427 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:16.106552 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:16.107830 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:16.108586 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:16.109828 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:16.110271 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:16.110440 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:16.113763 (+ 3323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:16.113776 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:16.113780 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:16.113813 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:16.116365 (+ 2552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:16.117087 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:16.117094 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:16.117104 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:16.117218 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:16.117606 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:16.117611 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:16.117614 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:16.117874 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:16.118028 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:16.118427 (+ 399us) server_negotiation.cc:299] Negotiation successful
0811 18:57:16.118818 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":191,"server-negotiator.queue_time_us":63}
I20260811 18:57:16.119160 1773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:16.105417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:39833 (local address 127.0.0.1:43844)
0811 18:57:16.106008 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:16.106023 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:16.106197 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:16.106616 (+ 419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:16.106620 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:16.107351 (+ 731us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:57:16.107680 (+ 329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:16.107691 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:16.108714 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:16.108721 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:16.109672 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:16.109684 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:16.110482 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:16.110488 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:16.110690 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:16.111301 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:57:16.111322 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:57:16.113572 (+ 2250us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:57:16.116510 (+ 2938us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:16.116525 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:16.116532 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:16.116953 (+ 421us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:16.117328 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:16.117333 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:16.117336 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:16.117487 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:16.118003 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:57:16.118010 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:57:16.118174 (+ 164us) client_negotiation.cc:769] Sending connection context
0811 18:57:16.118338 (+ 164us) client_negotiation.cc:240] Negotiation successful
0811 18:57:16.118684 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":339,"threads_started":1}
I20260811 18:57:16.124912 852 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.29.112.190@KRBTEST.COM'} at 127.0.0.1:43844:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 18:57:16.125896 852 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:16.132398 1344 tablet_service.cc:1467] Processing CreateTablet for tablet 311c5059f466463f94a22bf97177e12f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5c414de9a5ba403492c4ab7952aa4df2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 18:57:16.132398 1096 tablet_service.cc:1467] Processing CreateTablet for tablet 311c5059f466463f94a22bf97177e12f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5c414de9a5ba403492c4ab7952aa4df2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 18:57:16.132740 1586 tablet_service.cc:1467] Processing CreateTablet for tablet 311c5059f466463f94a22bf97177e12f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5c414de9a5ba403492c4ab7952aa4df2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 18:57:16.133425 1096 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 311c5059f466463f94a22bf97177e12f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.133420 1344 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 311c5059f466463f94a22bf97177e12f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.133581 1586 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 311c5059f466463f94a22bf97177e12f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.140161 1746 tablet_bootstrap.cc:492] T 311c5059f466463f94a22bf97177e12f P 4c0fb9abd83245b289dc12805882cc9e: Bootstrap starting.
I20260811 18:57:16.140864 1748 tablet_bootstrap.cc:492] T 311c5059f466463f94a22bf97177e12f P 937550b3337d417c9d5ec16caa6f3b09: Bootstrap starting.
I20260811 18:57:16.141479 1747 tablet_bootstrap.cc:492] T 311c5059f466463f94a22bf97177e12f P 482b0d44ef0941dd9f67913d024d29ba: Bootstrap starting.
I20260811 18:57:16.141952 1746 tablet_bootstrap.cc:654] T 311c5059f466463f94a22bf97177e12f P 4c0fb9abd83245b289dc12805882cc9e: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:16.142678 1748 tablet_bootstrap.cc:654] T 311c5059f466463f94a22bf97177e12f P 937550b3337d417c9d5ec16caa6f3b09: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:16.143356 1747 tablet_bootstrap.cc:654] T 311c5059f466463f94a22bf97177e12f P 482b0d44ef0941dd9f67913d024d29ba: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:16.153678 1746 tablet_bootstrap.cc:492] T 311c5059f466463f94a22bf97177e12f P 4c0fb9abd83245b289dc12805882cc9e: No bootstrap required, opened a new log
I20260811 18:57:16.153878 1746 ts_tablet_manager.cc:1397] T 311c5059f466463f94a22bf97177e12f P 4c0fb9abd83245b289dc12805882cc9e: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.010s
I20260811 18:57:16.154821 1746 raft_consensus.cc:348] T 311c5059f466463f94a22bf97177e12f P 4c0fb9abd83245b289dc12805882cc9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937550b3337d417c9d5ec16caa6f3b09" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 32865 } } peers { permanent_uuid: "482b0d44ef0941dd9f67913d024d29ba" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33619 } } peers { permanent_uuid: "4c0fb9abd83245b289dc12805882cc9e" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45211 } }
I20260811 18:57:16.155474 1747 tablet_bootstrap.cc:492] T 311c5059f466463f94a22bf97177e12f P 482b0d44ef0941dd9f67913d024d29ba: No bootstrap required, opened a new log
I20260811 18:57:16.155472 1746 raft_consensus.cc:374] T 311c5059f466463f94a22bf97177e12f P 4c0fb9abd83245b289dc12805882cc9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:16.155591 1746 raft_consensus.cc:729] T 311c5059f466463f94a22bf97177e12f P 4c0fb9abd83245b289dc12805882cc9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c0fb9abd83245b289dc12805882cc9e, State: Initialized, Role: FOLLOWER
I20260811 18:57:16.155643 1747 ts_tablet_manager.cc:1397] T 311c5059f466463f94a22bf97177e12f P 482b0d44ef0941dd9f67913d024d29ba: Time spent bootstrapping tablet: real 0.014s user 0.008s sys 0.004s
I20260811 18:57:16.155920 1746 consensus_queue.cc:260] T 311c5059f466463f94a22bf97177e12f P 4c0fb9abd83245b289dc12805882cc9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937550b3337d417c9d5ec16caa6f3b09" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 32865 } } peers { permanent_uuid: "482b0d44ef0941dd9f67913d024d29ba" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33619 } } peers { permanent_uuid: "4c0fb9abd83245b289dc12805882cc9e" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45211 } }
I20260811 18:57:16.156658 1746 ts_tablet_manager.cc:1428] T 311c5059f466463f94a22bf97177e12f P 4c0fb9abd83245b289dc12805882cc9e: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260811 18:57:16.156473 1753 tablet_replica.cc:439] T 311c5059f466463f94a22bf97177e12f P 4c0fb9abd83245b289dc12805882cc9e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937550b3337d417c9d5ec16caa6f3b09" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 32865 } } peers { permanent_uuid: "482b0d44ef0941dd9f67913d024d29ba" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33619 } } peers { permanent_uuid: "4c0fb9abd83245b289dc12805882cc9e" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45211 } } }
I20260811 18:57:16.156885 1753 tablet_replica.cc:442] T 311c5059f466463f94a22bf97177e12f P 4c0fb9abd83245b289dc12805882cc9e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:57:16.157575 1747 raft_consensus.cc:348] T 311c5059f466463f94a22bf97177e12f P 482b0d44ef0941dd9f67913d024d29ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937550b3337d417c9d5ec16caa6f3b09" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 32865 } } peers { permanent_uuid: "482b0d44ef0941dd9f67913d024d29ba" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33619 } } peers { permanent_uuid: "4c0fb9abd83245b289dc12805882cc9e" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45211 } }
I20260811 18:57:16.157853 1747 raft_consensus.cc:374] T 311c5059f466463f94a22bf97177e12f P 482b0d44ef0941dd9f67913d024d29ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:16.157954 1747 raft_consensus.cc:729] T 311c5059f466463f94a22bf97177e12f P 482b0d44ef0941dd9f67913d024d29ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482b0d44ef0941dd9f67913d024d29ba, State: Initialized, Role: FOLLOWER
I20260811 18:57:16.158257 1748 tablet_bootstrap.cc:492] T 311c5059f466463f94a22bf97177e12f P 937550b3337d417c9d5ec16caa6f3b09: No bootstrap required, opened a new log
I20260811 18:57:16.158222 1747 consensus_queue.cc:260] T 311c5059f466463f94a22bf97177e12f P 482b0d44ef0941dd9f67913d024d29ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937550b3337d417c9d5ec16caa6f3b09" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 32865 } } peers { permanent_uuid: "482b0d44ef0941dd9f67913d024d29ba" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33619 } } peers { permanent_uuid: "4c0fb9abd83245b289dc12805882cc9e" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45211 } }
I20260811 18:57:16.158444 1748 ts_tablet_manager.cc:1397] T 311c5059f466463f94a22bf97177e12f P 937550b3337d417c9d5ec16caa6f3b09: Time spent bootstrapping tablet: real 0.018s user 0.004s sys 0.008s
I20260811 18:57:16.159421 1748 raft_consensus.cc:348] T 311c5059f466463f94a22bf97177e12f P 937550b3337d417c9d5ec16caa6f3b09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937550b3337d417c9d5ec16caa6f3b09" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 32865 } } peers { permanent_uuid: "482b0d44ef0941dd9f67913d024d29ba" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33619 } } peers { permanent_uuid: "4c0fb9abd83245b289dc12805882cc9e" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45211 } }
I20260811 18:57:16.159511 1756 tablet_replica.cc:439] T 311c5059f466463f94a22bf97177e12f P 482b0d44ef0941dd9f67913d024d29ba: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937550b3337d417c9d5ec16caa6f3b09" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 32865 } } peers { permanent_uuid: "482b0d44ef0941dd9f67913d024d29ba" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33619 } } peers { permanent_uuid: "4c0fb9abd83245b289dc12805882cc9e" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45211 } } }
I20260811 18:57:16.159723 1748 raft_consensus.cc:374] T 311c5059f466463f94a22bf97177e12f P 937550b3337d417c9d5ec16caa6f3b09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:16.159773 1756 tablet_replica.cc:442] T 311c5059f466463f94a22bf97177e12f P 482b0d44ef0941dd9f67913d024d29ba: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:57:16.159838 1748 raft_consensus.cc:729] T 311c5059f466463f94a22bf97177e12f P 937550b3337d417c9d5ec16caa6f3b09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 937550b3337d417c9d5ec16caa6f3b09, State: Initialized, Role: FOLLOWER
I20260811 18:57:16.160110 1748 consensus_queue.cc:260] T 311c5059f466463f94a22bf97177e12f P 937550b3337d417c9d5ec16caa6f3b09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937550b3337d417c9d5ec16caa6f3b09" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 32865 } } peers { permanent_uuid: "482b0d44ef0941dd9f67913d024d29ba" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33619 } } peers { permanent_uuid: "4c0fb9abd83245b289dc12805882cc9e" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45211 } }
I20260811 18:57:16.160784 1758 tablet_replica.cc:439] T 311c5059f466463f94a22bf97177e12f P 937550b3337d417c9d5ec16caa6f3b09: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937550b3337d417c9d5ec16caa6f3b09" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 32865 } } peers { permanent_uuid: "482b0d44ef0941dd9f67913d024d29ba" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33619 } } peers { permanent_uuid: "4c0fb9abd83245b289dc12805882cc9e" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45211 } } }
I20260811 18:57:16.161186 1758 tablet_replica.cc:442] T 311c5059f466463f94a22bf97177e12f P 937550b3337d417c9d5ec16caa6f3b09: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:57:16.162742 1748 ts_tablet_manager.cc:1428] T 311c5059f466463f94a22bf97177e12f P 937550b3337d417c9d5ec16caa6f3b09: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260811 18:57:16.161453 1747 ts_tablet_manager.cc:1428] T 311c5059f466463f94a22bf97177e12f P 482b0d44ef0941dd9f67913d024d29ba: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
W20260811 18:57:16.170674 1468 tablet.cc:2369] T 311c5059f466463f94a22bf97177e12f P 482b0d44ef0941dd9f67913d024d29ba: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:57:16.265259 1753 raft_consensus.cc:482] T 311c5059f466463f94a22bf97177e12f P 4c0fb9abd83245b289dc12805882cc9e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:16.265651 1753 raft_consensus.cc:504] T 311c5059f466463f94a22bf97177e12f P 4c0fb9abd83245b289dc12805882cc9e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937550b3337d417c9d5ec16caa6f3b09" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 32865 } } peers { permanent_uuid: "482b0d44ef0941dd9f67913d024d29ba" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33619 } } peers { permanent_uuid: "4c0fb9abd83245b289dc12805882cc9e" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45211 } }
I20260811 18:57:16.267776 1753 leader_election.cc:288] T 311c5059f466463f94a22bf97177e12f P 4c0fb9abd83245b289dc12805882cc9e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 937550b3337d417c9d5ec16caa6f3b09 (127.29.112.129:32865), 482b0d44ef0941dd9f67913d024d29ba (127.29.112.130:33619)
I20260811 18:57:16.276084 1520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:16.268168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.130:33619 (local address 127.29.112.131:46189)
0811 18:57:16.268477 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:16.268515 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:16.268650 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:16.269078 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:16.269083 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:16.269116 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:57:16.269351 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:16.269365 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:16.270894 (+ 1529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:16.270920 (+ 26us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:16.273024 (+ 2104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:16.273039 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:16.275215 (+ 2176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:16.275220 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:16.275490 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:16.275524 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 18:57:16.275643 (+ 119us) client_negotiation.cc:240] Negotiation successful
0811 18:57:16.275785 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"mutex_wait_us":36}
I20260811 18:57:16.276233 1741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:16.268292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.131:46189 (local address 127.29.112.130:33619)
0811 18:57:16.268608 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:16.268619 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:16.268653 (+ 34us) server_negotiation.cc:407] Connection header received
0811 18:57:16.268802 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:16.268811 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:16.268913 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:16.269033 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:57:16.269499 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:16.270715 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:16.273256 (+ 2541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:16.274798 (+ 1542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:16.275002 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:16.275101 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:16.275713 (+ 612us) server_negotiation.cc:299] Negotiation successful
0811 18:57:16.275895 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147}
I20260811 18:57:16.277784 1745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:16.268482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.131:53129 (local address 127.29.112.129:32865)
0811 18:57:16.268695 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:16.268703 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:16.269278 (+ 575us) server_negotiation.cc:407] Connection header received
0811 18:57:16.269486 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:16.269492 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:16.269584 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:16.269711 (+ 127us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:57:16.270257 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:16.271584 (+ 1327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:16.274808 (+ 3224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:16.276334 (+ 1526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:16.276492 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:16.276563 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:16.277288 (+ 725us) server_negotiation.cc:299] Negotiation successful
0811 18:57:16.277480 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260811 18:57:16.277792 1779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:16.268256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:32865 (local address 127.29.112.131:53129)
0811 18:57:16.269177 (+ 921us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:16.269194 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:16.269320 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:16.269755 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:16.269759 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:16.269779 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:57:16.270104 (+ 325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:16.270112 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:16.271795 (+ 1683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:16.271804 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:16.274583 (+ 2779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:16.274605 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:16.276793 (+ 2188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:16.276801 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:16.277033 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:16.277085 (+ 52us) client_negotiation.cc:769] Sending connection context
0811 18:57:16.277241 (+ 156us) client_negotiation.cc:240] Negotiation successful
0811 18:57:16.277377 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":783,"mutex_wait_us":45,"thread_start_us":583,"threads_started":1}
I20260811 18:57:16.278466 1126 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "311c5059f466463f94a22bf97177e12f" candidate_uuid: "4c0fb9abd83245b289dc12805882cc9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "937550b3337d417c9d5ec16caa6f3b09" is_pre_election: true
I20260811 18:57:16.278774 1126 raft_consensus.cc:2454] T 311c5059f466463f94a22bf97177e12f P 937550b3337d417c9d5ec16caa6f3b09 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c0fb9abd83245b289dc12805882cc9e in term 0.
I20260811 18:57:16.279841 1510 leader_election.cc:302] T 311c5059f466463f94a22bf97177e12f P 4c0fb9abd83245b289dc12805882cc9e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c0fb9abd83245b289dc12805882cc9e, 937550b3337d417c9d5ec16caa6f3b09; no voters:
I20260811 18:57:16.279913 1375 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "311c5059f466463f94a22bf97177e12f" candidate_uuid: "4c0fb9abd83245b289dc12805882cc9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482b0d44ef0941dd9f67913d024d29ba" is_pre_election: true
I20260811 18:57:16.280313 1753 raft_consensus.cc:2781] T 311c5059f466463f94a22bf97177e12f P 4c0fb9abd83245b289dc12805882cc9e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:16.280462 1753 raft_consensus.cc:482] T 311c5059f466463f94a22bf97177e12f P 4c0fb9abd83245b289dc12805882cc9e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:16.280443 1375 raft_consensus.cc:2454] T 311c5059f466463f94a22bf97177e12f P 482b0d44ef0941dd9f67913d024d29ba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c0fb9abd83245b289dc12805882cc9e in term 0.
I20260811 18:57:16.280936 1753 raft_consensus.cc:3037] T 311c5059f466463f94a22bf97177e12f P 4c0fb9abd83245b289dc12805882cc9e [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:16.283656 1753 raft_consensus.cc:504] T 311c5059f466463f94a22bf97177e12f P 4c0fb9abd83245b289dc12805882cc9e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937550b3337d417c9d5ec16caa6f3b09" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 32865 } } peers { permanent_uuid: "482b0d44ef0941dd9f67913d024d29ba" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33619 } } peers { permanent_uuid: "4c0fb9abd83245b289dc12805882cc9e" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45211 } }
I20260811 18:57:16.284417 1753 leader_election.cc:288] T 311c5059f466463f94a22bf97177e12f P 4c0fb9abd83245b289dc12805882cc9e [CANDIDATE]: Term 1 election: Requested vote from peers 937550b3337d417c9d5ec16caa6f3b09 (127.29.112.129:32865), 482b0d44ef0941dd9f67913d024d29ba (127.29.112.130:33619)
I20260811 18:57:16.284929 1375 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "311c5059f466463f94a22bf97177e12f" candidate_uuid: "4c0fb9abd83245b289dc12805882cc9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482b0d44ef0941dd9f67913d024d29ba"
I20260811 18:57:16.284981 1126 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "311c5059f466463f94a22bf97177e12f" candidate_uuid: "4c0fb9abd83245b289dc12805882cc9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "937550b3337d417c9d5ec16caa6f3b09"
I20260811 18:57:16.285127 1375 raft_consensus.cc:3037] T 311c5059f466463f94a22bf97177e12f P 482b0d44ef0941dd9f67913d024d29ba [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:16.285295 1126 raft_consensus.cc:3037] T 311c5059f466463f94a22bf97177e12f P 937550b3337d417c9d5ec16caa6f3b09 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:16.288082 1375 raft_consensus.cc:2454] T 311c5059f466463f94a22bf97177e12f P 482b0d44ef0941dd9f67913d024d29ba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c0fb9abd83245b289dc12805882cc9e in term 1.
I20260811 18:57:16.288095 1126 raft_consensus.cc:2454] T 311c5059f466463f94a22bf97177e12f P 937550b3337d417c9d5ec16caa6f3b09 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c0fb9abd83245b289dc12805882cc9e in term 1.
I20260811 18:57:16.288647 1509 leader_election.cc:302] T 311c5059f466463f94a22bf97177e12f P 4c0fb9abd83245b289dc12805882cc9e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 482b0d44ef0941dd9f67913d024d29ba, 4c0fb9abd83245b289dc12805882cc9e; no voters:
I20260811 18:57:16.289024 1753 raft_consensus.cc:2781] T 311c5059f466463f94a22bf97177e12f P 4c0fb9abd83245b289dc12805882cc9e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:16.289640 1753 raft_consensus.cc:686] T 311c5059f466463f94a22bf97177e12f P 4c0fb9abd83245b289dc12805882cc9e [term 1 LEADER]: Becoming Leader. State: Replica: 4c0fb9abd83245b289dc12805882cc9e, State: Running, Role: LEADER
I20260811 18:57:16.290241 1753 consensus_queue.cc:237] T 311c5059f466463f94a22bf97177e12f P 4c0fb9abd83245b289dc12805882cc9e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937550b3337d417c9d5ec16caa6f3b09" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 32865 } } peers { permanent_uuid: "482b0d44ef0941dd9f67913d024d29ba" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33619 } } peers { permanent_uuid: "4c0fb9abd83245b289dc12805882cc9e" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45211 } }
I20260811 18:57:16.294157 1785 tablet_replica.cc:439] T 311c5059f466463f94a22bf97177e12f P 4c0fb9abd83245b289dc12805882cc9e: TxnStatusTablet state changed. Reason: New leader 4c0fb9abd83245b289dc12805882cc9e. Latest consensus state: current_term: 1 leader_uuid: "4c0fb9abd83245b289dc12805882cc9e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937550b3337d417c9d5ec16caa6f3b09" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 32865 } } peers { permanent_uuid: "482b0d44ef0941dd9f67913d024d29ba" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33619 } } peers { permanent_uuid: "4c0fb9abd83245b289dc12805882cc9e" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45211 } } }
I20260811 18:57:16.294612 1785 tablet_replica.cc:442] T 311c5059f466463f94a22bf97177e12f P 4c0fb9abd83245b289dc12805882cc9e: This TxnStatusTablet replica's current role is: LEADER
I20260811 18:57:16.295526 1787 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 18:57:16.295831 1787 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 18:57:16.296272 853 catalog_manager.cc:5495] T 311c5059f466463f94a22bf97177e12f P 4c0fb9abd83245b289dc12805882cc9e reported cstate change: term changed from 0 to 1, leader changed from <none> to 4c0fb9abd83245b289dc12805882cc9e (127.29.112.131). New cstate: current_term: 1 leader_uuid: "4c0fb9abd83245b289dc12805882cc9e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937550b3337d417c9d5ec16caa6f3b09" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 32865 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "482b0d44ef0941dd9f67913d024d29ba" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c0fb9abd83245b289dc12805882cc9e" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45211 } health_report { overall_health: HEALTHY } } }
I20260811 18:57:16.317706 1773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:16.311692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.131:45211 (local address 127.0.0.1:47640)
0811 18:57:16.312066 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:16.312083 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:16.312256 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:16.312746 (+ 490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:16.312751 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:16.312763 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 18:57:16.312960 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:16.312969 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:16.314103 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:16.314109 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:16.315214 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:16.315227 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:16.316001 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:16.316006 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:16.316191 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:16.316247 (+ 56us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 18:57:16.317100 (+ 853us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 18:57:16.317121 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 18:57:16.317241 (+ 120us) client_negotiation.cc:240] Negotiation successful
0811 18:57:16.317406 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229}
I20260811 18:57:16.317716 1740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:16.311679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47640 (local address 127.29.112.131:45211)
0811 18:57:16.311875 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:16.311887 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:16.312164 (+ 277us) server_negotiation.cc:407] Connection header received
0811 18:57:16.312465 (+ 301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:16.312469 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:16.312575 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:16.312725 (+ 150us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:57:16.313111 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:16.313965 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:16.315357 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:16.315829 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:16.315991 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:16.316445 (+ 454us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:57:16.316979 (+ 534us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:57:16.317154 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:16.317266 (+ 112us) server_negotiation.cc:299] Negotiation successful
0811 18:57:16.317360 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 18:57:16.326268 1375 raft_consensus.cc:1261] T 311c5059f466463f94a22bf97177e12f P 482b0d44ef0941dd9f67913d024d29ba [term 1 FOLLOWER]: Refusing update from remote peer 4c0fb9abd83245b289dc12805882cc9e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:57:16.326443 1126 raft_consensus.cc:1261] T 311c5059f466463f94a22bf97177e12f P 937550b3337d417c9d5ec16caa6f3b09 [term 1 FOLLOWER]: Refusing update from remote peer 4c0fb9abd83245b289dc12805882cc9e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:57:16.327378 1785 consensus_queue.cc:1035] T 311c5059f466463f94a22bf97177e12f P 4c0fb9abd83245b289dc12805882cc9e [LEADER]: Connected to new peer: Peer: permanent_uuid: "482b0d44ef0941dd9f67913d024d29ba" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:16.327703 1753 consensus_queue.cc:1035] T 311c5059f466463f94a22bf97177e12f P 4c0fb9abd83245b289dc12805882cc9e [LEADER]: Connected to new peer: Peer: permanent_uuid: "937550b3337d417c9d5ec16caa6f3b09" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 32865 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:16.339174 1756 tablet_replica.cc:439] T 311c5059f466463f94a22bf97177e12f P 482b0d44ef0941dd9f67913d024d29ba: TxnStatusTablet state changed. Reason: New leader 4c0fb9abd83245b289dc12805882cc9e. Latest consensus state: current_term: 1 leader_uuid: "4c0fb9abd83245b289dc12805882cc9e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937550b3337d417c9d5ec16caa6f3b09" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 32865 } } peers { permanent_uuid: "482b0d44ef0941dd9f67913d024d29ba" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33619 } } peers { permanent_uuid: "4c0fb9abd83245b289dc12805882cc9e" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45211 } } }
I20260811 18:57:16.339514 1756 tablet_replica.cc:442] T 311c5059f466463f94a22bf97177e12f P 482b0d44ef0941dd9f67913d024d29ba: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:57:16.341620 1796 mvcc.cc:205] Tried to move back new op lower bound from 7317400110382436352 to 7317400110249533440. Current Snapshot: MvccSnapshot[applied={T|T < 7317400110382436352}]
I20260811 18:57:16.344798 1753 tablet_replica.cc:439] T 311c5059f466463f94a22bf97177e12f P 4c0fb9abd83245b289dc12805882cc9e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4c0fb9abd83245b289dc12805882cc9e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937550b3337d417c9d5ec16caa6f3b09" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 32865 } } peers { permanent_uuid: "482b0d44ef0941dd9f67913d024d29ba" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33619 } } peers { permanent_uuid: "4c0fb9abd83245b289dc12805882cc9e" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45211 } } }
I20260811 18:57:16.345089 1753 tablet_replica.cc:442] T 311c5059f466463f94a22bf97177e12f P 4c0fb9abd83245b289dc12805882cc9e: This TxnStatusTablet replica's current role is: LEADER
I20260811 18:57:16.346447 1758 tablet_replica.cc:439] T 311c5059f466463f94a22bf97177e12f P 937550b3337d417c9d5ec16caa6f3b09: TxnStatusTablet state changed. Reason: New leader 4c0fb9abd83245b289dc12805882cc9e. Latest consensus state: current_term: 1 leader_uuid: "4c0fb9abd83245b289dc12805882cc9e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937550b3337d417c9d5ec16caa6f3b09" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 32865 } } peers { permanent_uuid: "482b0d44ef0941dd9f67913d024d29ba" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33619 } } peers { permanent_uuid: "4c0fb9abd83245b289dc12805882cc9e" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45211 } } }
I20260811 18:57:16.346781 1758 tablet_replica.cc:442] T 311c5059f466463f94a22bf97177e12f P 937550b3337d417c9d5ec16caa6f3b09: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:57:16.350121 1756 tablet_replica.cc:439] T 311c5059f466463f94a22bf97177e12f P 482b0d44ef0941dd9f67913d024d29ba: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4c0fb9abd83245b289dc12805882cc9e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937550b3337d417c9d5ec16caa6f3b09" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 32865 } } peers { permanent_uuid: "482b0d44ef0941dd9f67913d024d29ba" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33619 } } peers { permanent_uuid: "4c0fb9abd83245b289dc12805882cc9e" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45211 } } }
I20260811 18:57:16.350452 1756 tablet_replica.cc:442] T 311c5059f466463f94a22bf97177e12f P 482b0d44ef0941dd9f67913d024d29ba: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:57:16.352437 1785 tablet_replica.cc:439] T 311c5059f466463f94a22bf97177e12f P 4c0fb9abd83245b289dc12805882cc9e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4c0fb9abd83245b289dc12805882cc9e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937550b3337d417c9d5ec16caa6f3b09" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 32865 } } peers { permanent_uuid: "482b0d44ef0941dd9f67913d024d29ba" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33619 } } peers { permanent_uuid: "4c0fb9abd83245b289dc12805882cc9e" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45211 } } }
I20260811 18:57:16.352954 1785 tablet_replica.cc:442] T 311c5059f466463f94a22bf97177e12f P 4c0fb9abd83245b289dc12805882cc9e: This TxnStatusTablet replica's current role is: LEADER
I20260811 18:57:16.361361 1801 mvcc.cc:205] Tried to move back new op lower bound from 7317400110382436352 to 7317400110249533440. Current Snapshot: MvccSnapshot[applied={T|T < 7317400110382436352}]
I20260811 18:57:16.368207 1758 tablet_replica.cc:439] T 311c5059f466463f94a22bf97177e12f P 937550b3337d417c9d5ec16caa6f3b09: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4c0fb9abd83245b289dc12805882cc9e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937550b3337d417c9d5ec16caa6f3b09" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 32865 } } peers { permanent_uuid: "482b0d44ef0941dd9f67913d024d29ba" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33619 } } peers { permanent_uuid: "4c0fb9abd83245b289dc12805882cc9e" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45211 } } }
I20260811 18:57:16.368480 1758 tablet_replica.cc:442] T 311c5059f466463f94a22bf97177e12f P 937550b3337d417c9d5ec16caa6f3b09: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:57:16.378211 1741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:16.371707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38928 (local address 127.29.112.130:33619)
0811 18:57:16.371877 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:16.371884 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:16.371902 (+ 18us) server_negotiation.cc:407] Connection header received
0811 18:57:16.372005 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:16.372010 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:16.372124 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:16.372291 (+ 167us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:57:16.372699 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:16.373605 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:16.375938 (+ 2333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:16.376358 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:16.376467 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:16.377010 (+ 543us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:57:16.377542 (+ 532us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:57:16.377647 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:16.377822 (+ 175us) server_negotiation.cc:299] Negotiation successful
0811 18:57:16.377890 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 18:57:16.397370 1818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:16.391291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.131:45211 (local address 127.29.112.130:33057)
0811 18:57:16.392018 (+ 727us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:16.392034 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:16.392248 (+ 214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:16.392620 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:16.392624 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:16.392636 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 18:57:16.392997 (+ 361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:16.393009 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:16.394081 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:16.394088 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:16.395273 (+ 1185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:16.395285 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:16.396091 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:16.396098 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:16.396233 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:16.396288 (+ 55us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 18:57:16.396706 (+ 418us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 18:57:16.396720 (+ 14us) client_negotiation.cc:769] Sending connection context
0811 18:57:16.396816 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 18:57:16.396970 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":578,"thread_start_us":343,"threads_started":1}
I20260811 18:57:16.397349 1740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:16.391292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.130:33057 (local address 127.29.112.131:45211)
0811 18:57:16.391472 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:16.391477 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:16.392121 (+ 644us) server_negotiation.cc:407] Connection header received
0811 18:57:16.392431 (+ 310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:16.392435 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:16.392503 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:16.392605 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:57:16.393164 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:16.393938 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:16.395414 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:16.395894 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:16.396030 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:16.396497 (+ 467us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:57:16.396598 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:57:16.396703 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:16.396855 (+ 152us) server_negotiation.cc:299] Negotiation successful
0811 18:57:16.396940 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 18:57:16.409142 1126 raft_consensus.cc:1261] T be0213a1e743498d9455d9807a69f0b2 P 937550b3337d417c9d5ec16caa6f3b09 [term 1 FOLLOWER]: Refusing update from remote peer 482b0d44ef0941dd9f67913d024d29ba: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:57:16.409355 1636 raft_consensus.cc:1261] T be0213a1e743498d9455d9807a69f0b2 P 4c0fb9abd83245b289dc12805882cc9e [term 1 FOLLOWER]: Refusing update from remote peer 482b0d44ef0941dd9f67913d024d29ba: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:57:16.409848 1763 consensus_queue.cc:1035] T be0213a1e743498d9455d9807a69f0b2 P 482b0d44ef0941dd9f67913d024d29ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "937550b3337d417c9d5ec16caa6f3b09" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 32865 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:16.410765 1763 consensus_queue.cc:1035] T be0213a1e743498d9455d9807a69f0b2 P 482b0d44ef0941dd9f67913d024d29ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c0fb9abd83245b289dc12805882cc9e" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45211 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:16.416788 1799 mvcc.cc:205] Tried to move back new op lower bound from 7317400110725349376 to 7317400109222326272. Current Snapshot: MvccSnapshot[applied={T|T < 7317400110725349376}]
I20260811 18:57:16.473745 1741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:16.452977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.131:35207 (local address 127.29.112.130:33619)
0811 18:57:16.454179 (+ 1202us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:16.454187 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:16.466659 (+ 12472us) server_negotiation.cc:407] Connection header received
0811 18:57:16.466751 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:16.466756 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:16.466845 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:16.466929 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:57:16.467988 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:16.469161 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:16.471369 (+ 2208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:16.472026 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:16.472162 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:16.472940 (+ 778us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:57:16.473079 (+ 139us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:57:16.473142 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:16.473382 (+ 240us) server_negotiation.cc:299] Negotiation successful
0811 18:57:16.473467 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117}
I20260811 18:57:16.474835 1829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:16.453068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.130:33619 (local address 127.29.112.131:35207)
0811 18:57:16.463847 (+ 10779us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:16.463869 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:16.464033 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:16.467373 (+ 3340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:16.467379 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:16.467393 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 18:57:16.467813 (+ 420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:16.467826 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:16.469402 (+ 1576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:16.469407 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:16.471215 (+ 1808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:16.471231 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:16.472434 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:16.472441 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:16.472622 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:16.472701 (+ 79us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 18:57:16.473225 (+ 524us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 18:57:16.473236 (+ 11us) client_negotiation.cc:769] Sending connection context
0811 18:57:16.474203 (+ 967us) client_negotiation.cc:240] Negotiation successful
0811 18:57:16.474374 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10560,"thread_start_us":11057,"threads_started":1}
I20260811 18:57:16.525225 853 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43842:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:57:16.525542 853 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43842:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:57:16.530224 853 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 50c83e7e69974b90a46fe19124750a8d: Sending DeleteTablet for 3 replicas of tablet be0213a1e743498d9455d9807a69f0b2
I20260811 18:57:16.531523 1344 tablet_service.cc:1514] Processing DeleteTablet for tablet be0213a1e743498d9455d9807a69f0b2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:16 UTC) from {username='kudu', principal='kudu/127.29.112.190@KRBTEST.COM'} at 127.0.0.1:38916
I20260811 18:57:16.531605 1586 tablet_service.cc:1514] Processing DeleteTablet for tablet be0213a1e743498d9455d9807a69f0b2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:16 UTC) from {username='kudu', principal='kudu/127.29.112.190@KRBTEST.COM'} at 127.0.0.1:47630
I20260811 18:57:16.532816 1096 tablet_service.cc:1514] Processing DeleteTablet for tablet be0213a1e743498d9455d9807a69f0b2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:16 UTC) from {username='kudu', principal='kudu/127.29.112.190@KRBTEST.COM'} at 127.0.0.1:57678
I20260811 18:57:16.541699 853 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43842:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:16.543001 1842 tablet_replica.cc:330] T be0213a1e743498d9455d9807a69f0b2 P 937550b3337d417c9d5ec16caa6f3b09: stopping tablet replica
W20260811 18:57:16.543212 853 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:16.543483 1842 raft_consensus.cc:2229] T be0213a1e743498d9455d9807a69f0b2 P 937550b3337d417c9d5ec16caa6f3b09 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:16.543916 1842 raft_consensus.cc:2258] T be0213a1e743498d9455d9807a69f0b2 P 937550b3337d417c9d5ec16caa6f3b09 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:16.546443 1842 ts_tablet_manager.cc:1905] T be0213a1e743498d9455d9807a69f0b2 P 937550b3337d417c9d5ec16caa6f3b09: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:16.551836 1341 tablet_service.cc:1467] Processing CreateTablet for tablet fceef53720a24ab1b44711bab8a69d4b (DEFAULT_TABLE table=test-table [id=2d4e5f51a6834c89b7eb1f9cff740dce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.552418 1341 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fceef53720a24ab1b44711bab8a69d4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.552553 1584 tablet_service.cc:1467] Processing CreateTablet for tablet fceef53720a24ab1b44711bab8a69d4b (DEFAULT_TABLE table=test-table [id=2d4e5f51a6834c89b7eb1f9cff740dce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.553107 1584 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fceef53720a24ab1b44711bab8a69d4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.555859 1841 tablet_replica.cc:330] T be0213a1e743498d9455d9807a69f0b2 P 4c0fb9abd83245b289dc12805882cc9e: stopping tablet replica
I20260811 18:57:16.556485 1841 raft_consensus.cc:2229] T be0213a1e743498d9455d9807a69f0b2 P 4c0fb9abd83245b289dc12805882cc9e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:16.556970 1841 raft_consensus.cc:2258] T be0213a1e743498d9455d9807a69f0b2 P 4c0fb9abd83245b289dc12805882cc9e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:16.559072 1096 tablet_service.cc:1467] Processing CreateTablet for tablet fceef53720a24ab1b44711bab8a69d4b (DEFAULT_TABLE table=test-table [id=2d4e5f51a6834c89b7eb1f9cff740dce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:16.559578 1096 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fceef53720a24ab1b44711bab8a69d4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:16.559957 1841 ts_tablet_manager.cc:1905] T be0213a1e743498d9455d9807a69f0b2 P 4c0fb9abd83245b289dc12805882cc9e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:16.560614 1746 tablet_bootstrap.cc:492] T fceef53720a24ab1b44711bab8a69d4b P 4c0fb9abd83245b289dc12805882cc9e: Bootstrap starting.
I20260811 18:57:16.562489 1746 tablet_bootstrap.cc:654] T fceef53720a24ab1b44711bab8a69d4b P 4c0fb9abd83245b289dc12805882cc9e: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:16.562965 1842 ts_tablet_manager.cc:1918] T be0213a1e743498d9455d9807a69f0b2 P 937550b3337d417c9d5ec16caa6f3b09: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 18:57:16.563166 1842 log.cc:1196] T be0213a1e743498d9455d9807a69f0b2 P 937550b3337d417c9d5ec16caa6f3b09: Deleting WAL directory at /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/minicluster-data/ts-0/wal/wals/be0213a1e743498d9455d9807a69f0b2
I20260811 18:57:16.563802 1842 ts_tablet_manager.cc:1939] T be0213a1e743498d9455d9807a69f0b2 P 937550b3337d417c9d5ec16caa6f3b09: Deleting consensus metadata
I20260811 18:57:16.565989 829 catalog_manager.cc:4841] TS 937550b3337d417c9d5ec16caa6f3b09 (127.29.112.129:32865): tablet be0213a1e743498d9455d9807a69f0b2 (table test-table [id=32ca67adae6f4758a8e46eadc36bc92c]) successfully deleted
I20260811 18:57:16.570001 1747 tablet_bootstrap.cc:492] T fceef53720a24ab1b44711bab8a69d4b P 482b0d44ef0941dd9f67913d024d29ba: Bootstrap starting.
I20260811 18:57:16.571465 1841 ts_tablet_manager.cc:1918] T be0213a1e743498d9455d9807a69f0b2 P 4c0fb9abd83245b289dc12805882cc9e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 18:57:16.571728 1841 log.cc:1196] T be0213a1e743498d9455d9807a69f0b2 P 4c0fb9abd83245b289dc12805882cc9e: Deleting WAL directory at /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/minicluster-data/ts-2/wal/wals/be0213a1e743498d9455d9807a69f0b2
I20260811 18:57:16.571761 1747 tablet_bootstrap.cc:654] T fceef53720a24ab1b44711bab8a69d4b P 482b0d44ef0941dd9f67913d024d29ba: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:16.572330 1841 ts_tablet_manager.cc:1939] T be0213a1e743498d9455d9807a69f0b2 P 4c0fb9abd83245b289dc12805882cc9e: Deleting consensus metadata
I20260811 18:57:16.574262 825 catalog_manager.cc:4841] TS 4c0fb9abd83245b289dc12805882cc9e (127.29.112.131:45211): tablet be0213a1e743498d9455d9807a69f0b2 (table test-table [id=32ca67adae6f4758a8e46eadc36bc92c]) successfully deleted
I20260811 18:57:16.574489 1748 tablet_bootstrap.cc:492] T fceef53720a24ab1b44711bab8a69d4b P 937550b3337d417c9d5ec16caa6f3b09: Bootstrap starting.
I20260811 18:57:16.576357 1834 tablet_replica.cc:330] T be0213a1e743498d9455d9807a69f0b2 P 482b0d44ef0941dd9f67913d024d29ba: stopping tablet replica
I20260811 18:57:16.576479 1748 tablet_bootstrap.cc:654] T fceef53720a24ab1b44711bab8a69d4b P 937550b3337d417c9d5ec16caa6f3b09: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:16.576857 1834 raft_consensus.cc:2229] T be0213a1e743498d9455d9807a69f0b2 P 482b0d44ef0941dd9f67913d024d29ba [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:16.577522 1834 raft_consensus.cc:2258] T be0213a1e743498d9455d9807a69f0b2 P 482b0d44ef0941dd9f67913d024d29ba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:16.578950 1748 tablet_bootstrap.cc:492] T fceef53720a24ab1b44711bab8a69d4b P 937550b3337d417c9d5ec16caa6f3b09: No bootstrap required, opened a new log
I20260811 18:57:16.579492 1746 tablet_bootstrap.cc:492] T fceef53720a24ab1b44711bab8a69d4b P 4c0fb9abd83245b289dc12805882cc9e: No bootstrap required, opened a new log
I20260811 18:57:16.579676 1746 ts_tablet_manager.cc:1397] T fceef53720a24ab1b44711bab8a69d4b P 4c0fb9abd83245b289dc12805882cc9e: Time spent bootstrapping tablet: real 0.019s user 0.003s sys 0.004s
I20260811 18:57:16.580505 1748 ts_tablet_manager.cc:1397] T fceef53720a24ab1b44711bab8a69d4b P 937550b3337d417c9d5ec16caa6f3b09: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 18:57:16.580528 1834 ts_tablet_manager.cc:1905] T be0213a1e743498d9455d9807a69f0b2 P 482b0d44ef0941dd9f67913d024d29ba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:16.580535 1746 raft_consensus.cc:348] T fceef53720a24ab1b44711bab8a69d4b P 4c0fb9abd83245b289dc12805882cc9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937550b3337d417c9d5ec16caa6f3b09" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 32865 } } peers { permanent_uuid: "482b0d44ef0941dd9f67913d024d29ba" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33619 } } peers { permanent_uuid: "4c0fb9abd83245b289dc12805882cc9e" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45211 } }
I20260811 18:57:16.580798 1746 raft_consensus.cc:374] T fceef53720a24ab1b44711bab8a69d4b P 4c0fb9abd83245b289dc12805882cc9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:16.580893 1746 raft_consensus.cc:729] T fceef53720a24ab1b44711bab8a69d4b P 4c0fb9abd83245b289dc12805882cc9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c0fb9abd83245b289dc12805882cc9e, State: Initialized, Role: FOLLOWER
I20260811 18:57:16.581568 1748 raft_consensus.cc:348] T fceef53720a24ab1b44711bab8a69d4b P 937550b3337d417c9d5ec16caa6f3b09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937550b3337d417c9d5ec16caa6f3b09" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 32865 } } peers { permanent_uuid: "482b0d44ef0941dd9f67913d024d29ba" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33619 } } peers { permanent_uuid: "4c0fb9abd83245b289dc12805882cc9e" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45211 } }
I20260811 18:57:16.581946 1748 raft_consensus.cc:374] T fceef53720a24ab1b44711bab8a69d4b P 937550b3337d417c9d5ec16caa6f3b09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:16.582083 1748 raft_consensus.cc:729] T fceef53720a24ab1b44711bab8a69d4b P 937550b3337d417c9d5ec16caa6f3b09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 937550b3337d417c9d5ec16caa6f3b09, State: Initialized, Role: FOLLOWER
I20260811 18:57:16.582449 1748 consensus_queue.cc:260] T fceef53720a24ab1b44711bab8a69d4b P 937550b3337d417c9d5ec16caa6f3b09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937550b3337d417c9d5ec16caa6f3b09" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 32865 } } peers { permanent_uuid: "482b0d44ef0941dd9f67913d024d29ba" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33619 } } peers { permanent_uuid: "4c0fb9abd83245b289dc12805882cc9e" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45211 } }
I20260811 18:57:16.582669 1746 consensus_queue.cc:260] T fceef53720a24ab1b44711bab8a69d4b P 4c0fb9abd83245b289dc12805882cc9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937550b3337d417c9d5ec16caa6f3b09" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 32865 } } peers { permanent_uuid: "482b0d44ef0941dd9f67913d024d29ba" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33619 } } peers { permanent_uuid: "4c0fb9abd83245b289dc12805882cc9e" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45211 } }
I20260811 18:57:16.583329 1746 ts_tablet_manager.cc:1428] T fceef53720a24ab1b44711bab8a69d4b P 4c0fb9abd83245b289dc12805882cc9e: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260811 18:57:16.583350 1748 ts_tablet_manager.cc:1428] T fceef53720a24ab1b44711bab8a69d4b P 937550b3337d417c9d5ec16caa6f3b09: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260811 18:57:16.592895 1747 tablet_bootstrap.cc:492] T fceef53720a24ab1b44711bab8a69d4b P 482b0d44ef0941dd9f67913d024d29ba: No bootstrap required, opened a new log
I20260811 18:57:16.593124 1747 ts_tablet_manager.cc:1397] T fceef53720a24ab1b44711bab8a69d4b P 482b0d44ef0941dd9f67913d024d29ba: Time spent bootstrapping tablet: real 0.023s user 0.010s sys 0.000s
I20260811 18:57:16.594085 1747 raft_consensus.cc:348] T fceef53720a24ab1b44711bab8a69d4b P 482b0d44ef0941dd9f67913d024d29ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937550b3337d417c9d5ec16caa6f3b09" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 32865 } } peers { permanent_uuid: "482b0d44ef0941dd9f67913d024d29ba" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33619 } } peers { permanent_uuid: "4c0fb9abd83245b289dc12805882cc9e" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45211 } }
I20260811 18:57:16.594377 1747 raft_consensus.cc:374] T fceef53720a24ab1b44711bab8a69d4b P 482b0d44ef0941dd9f67913d024d29ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:16.594476 1747 raft_consensus.cc:729] T fceef53720a24ab1b44711bab8a69d4b P 482b0d44ef0941dd9f67913d024d29ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482b0d44ef0941dd9f67913d024d29ba, State: Initialized, Role: FOLLOWER
I20260811 18:57:16.594832 1747 consensus_queue.cc:260] T fceef53720a24ab1b44711bab8a69d4b P 482b0d44ef0941dd9f67913d024d29ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937550b3337d417c9d5ec16caa6f3b09" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 32865 } } peers { permanent_uuid: "482b0d44ef0941dd9f67913d024d29ba" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33619 } } peers { permanent_uuid: "4c0fb9abd83245b289dc12805882cc9e" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45211 } }
I20260811 18:57:16.595021 1834 ts_tablet_manager.cc:1918] T be0213a1e743498d9455d9807a69f0b2 P 482b0d44ef0941dd9f67913d024d29ba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 18:57:16.595206 1834 log.cc:1196] T be0213a1e743498d9455d9807a69f0b2 P 482b0d44ef0941dd9f67913d024d29ba: Deleting WAL directory at /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/minicluster-data/ts-1/wal/wals/be0213a1e743498d9455d9807a69f0b2
I20260811 18:57:16.596464 1834 ts_tablet_manager.cc:1939] T be0213a1e743498d9455d9807a69f0b2 P 482b0d44ef0941dd9f67913d024d29ba: Deleting consensus metadata
I20260811 18:57:16.595642 1747 ts_tablet_manager.cc:1428] T fceef53720a24ab1b44711bab8a69d4b P 482b0d44ef0941dd9f67913d024d29ba: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260811 18:57:16.598485 826 catalog_manager.cc:4841] TS 482b0d44ef0941dd9f67913d024d29ba (127.29.112.130:33619): tablet be0213a1e743498d9455d9807a69f0b2 (table test-table [id=32ca67adae6f4758a8e46eadc36bc92c]) successfully deleted
I20260811 18:57:16.748342 1831 raft_consensus.cc:482] T fceef53720a24ab1b44711bab8a69d4b P 4c0fb9abd83245b289dc12805882cc9e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:16.748616 1831 raft_consensus.cc:504] T fceef53720a24ab1b44711bab8a69d4b P 4c0fb9abd83245b289dc12805882cc9e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937550b3337d417c9d5ec16caa6f3b09" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 32865 } } peers { permanent_uuid: "482b0d44ef0941dd9f67913d024d29ba" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33619 } } peers { permanent_uuid: "4c0fb9abd83245b289dc12805882cc9e" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45211 } }
I20260811 18:57:16.749553 1831 leader_election.cc:288] T fceef53720a24ab1b44711bab8a69d4b P 4c0fb9abd83245b289dc12805882cc9e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 937550b3337d417c9d5ec16caa6f3b09 (127.29.112.129:32865), 482b0d44ef0941dd9f67913d024d29ba (127.29.112.130:33619)
I20260811 18:57:16.750624 1370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fceef53720a24ab1b44711bab8a69d4b" candidate_uuid: "4c0fb9abd83245b289dc12805882cc9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482b0d44ef0941dd9f67913d024d29ba" is_pre_election: true
I20260811 18:57:16.750952 1370 raft_consensus.cc:2454] T fceef53720a24ab1b44711bab8a69d4b P 482b0d44ef0941dd9f67913d024d29ba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c0fb9abd83245b289dc12805882cc9e in term 0.
I20260811 18:57:16.751950 1126 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fceef53720a24ab1b44711bab8a69d4b" candidate_uuid: "4c0fb9abd83245b289dc12805882cc9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "937550b3337d417c9d5ec16caa6f3b09" is_pre_election: true
I20260811 18:57:16.752202 1126 raft_consensus.cc:2454] T fceef53720a24ab1b44711bab8a69d4b P 937550b3337d417c9d5ec16caa6f3b09 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c0fb9abd83245b289dc12805882cc9e in term 0.
I20260811 18:57:16.752975 1509 leader_election.cc:302] T fceef53720a24ab1b44711bab8a69d4b P 4c0fb9abd83245b289dc12805882cc9e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 482b0d44ef0941dd9f67913d024d29ba, 4c0fb9abd83245b289dc12805882cc9e; no voters:
I20260811 18:57:16.753552 1831 raft_consensus.cc:2781] T fceef53720a24ab1b44711bab8a69d4b P 4c0fb9abd83245b289dc12805882cc9e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:16.753696 1831 raft_consensus.cc:482] T fceef53720a24ab1b44711bab8a69d4b P 4c0fb9abd83245b289dc12805882cc9e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:16.753779 1831 raft_consensus.cc:3037] T fceef53720a24ab1b44711bab8a69d4b P 4c0fb9abd83245b289dc12805882cc9e [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:16.757256 1831 raft_consensus.cc:504] T fceef53720a24ab1b44711bab8a69d4b P 4c0fb9abd83245b289dc12805882cc9e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937550b3337d417c9d5ec16caa6f3b09" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 32865 } } peers { permanent_uuid: "482b0d44ef0941dd9f67913d024d29ba" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33619 } } peers { permanent_uuid: "4c0fb9abd83245b289dc12805882cc9e" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45211 } }
I20260811 18:57:16.758688 1126 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fceef53720a24ab1b44711bab8a69d4b" candidate_uuid: "4c0fb9abd83245b289dc12805882cc9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "937550b3337d417c9d5ec16caa6f3b09"
I20260811 18:57:16.758937 1126 raft_consensus.cc:3037] T fceef53720a24ab1b44711bab8a69d4b P 937550b3337d417c9d5ec16caa6f3b09 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:16.763947 1126 raft_consensus.cc:2454] T fceef53720a24ab1b44711bab8a69d4b P 937550b3337d417c9d5ec16caa6f3b09 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c0fb9abd83245b289dc12805882cc9e in term 1.
I20260811 18:57:16.764547 1510 leader_election.cc:302] T fceef53720a24ab1b44711bab8a69d4b P 4c0fb9abd83245b289dc12805882cc9e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c0fb9abd83245b289dc12805882cc9e, 937550b3337d417c9d5ec16caa6f3b09; no voters:
I20260811 18:57:16.764865 1785 raft_consensus.cc:2781] T fceef53720a24ab1b44711bab8a69d4b P 4c0fb9abd83245b289dc12805882cc9e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:16.765033 1785 raft_consensus.cc:686] T fceef53720a24ab1b44711bab8a69d4b P 4c0fb9abd83245b289dc12805882cc9e [term 1 LEADER]: Becoming Leader. State: Replica: 4c0fb9abd83245b289dc12805882cc9e, State: Running, Role: LEADER
I20260811 18:57:16.765303 1785 consensus_queue.cc:237] T fceef53720a24ab1b44711bab8a69d4b P 4c0fb9abd83245b289dc12805882cc9e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937550b3337d417c9d5ec16caa6f3b09" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 32865 } } peers { permanent_uuid: "482b0d44ef0941dd9f67913d024d29ba" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33619 } } peers { permanent_uuid: "4c0fb9abd83245b289dc12805882cc9e" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45211 } }
I20260811 18:57:16.769102 1370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fceef53720a24ab1b44711bab8a69d4b" candidate_uuid: "4c0fb9abd83245b289dc12805882cc9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482b0d44ef0941dd9f67913d024d29ba"
I20260811 18:57:16.769318 1370 raft_consensus.cc:3037] T fceef53720a24ab1b44711bab8a69d4b P 482b0d44ef0941dd9f67913d024d29ba [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:16.770874 1831 leader_election.cc:288] T fceef53720a24ab1b44711bab8a69d4b P 4c0fb9abd83245b289dc12805882cc9e [CANDIDATE]: Term 1 election: Requested vote from peers 937550b3337d417c9d5ec16caa6f3b09 (127.29.112.129:32865), 482b0d44ef0941dd9f67913d024d29ba (127.29.112.130:33619)
I20260811 18:57:16.772447 1370 raft_consensus.cc:2454] T fceef53720a24ab1b44711bab8a69d4b P 482b0d44ef0941dd9f67913d024d29ba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c0fb9abd83245b289dc12805882cc9e in term 1.
I20260811 18:57:16.775192 853 catalog_manager.cc:5495] T fceef53720a24ab1b44711bab8a69d4b P 4c0fb9abd83245b289dc12805882cc9e reported cstate change: term changed from 0 to 1, leader changed from <none> to 4c0fb9abd83245b289dc12805882cc9e (127.29.112.131). New cstate: current_term: 1 leader_uuid: "4c0fb9abd83245b289dc12805882cc9e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937550b3337d417c9d5ec16caa6f3b09" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 32865 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "482b0d44ef0941dd9f67913d024d29ba" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c0fb9abd83245b289dc12805882cc9e" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45211 } health_report { overall_health: HEALTHY } } }
I20260811 18:57:16.842036 1740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:16.831246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47646 (local address 127.29.112.131:45211)
0811 18:57:16.831499 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:16.831508 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:16.831537 (+ 29us) server_negotiation.cc:407] Connection header received
0811 18:57:16.831622 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:16.831627 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:16.831737 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:16.831887 (+ 150us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:57:16.832609 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:16.834013 (+ 1404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:16.836811 (+ 2798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:16.837623 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:16.837757 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:16.838898 (+ 1141us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:57:16.839100 (+ 202us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:57:16.839209 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:16.841548 (+ 2339us) server_negotiation.cc:299] Negotiation successful
0811 18:57:16.841643 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 18:57:16.865600 1740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:16.858665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.131:59379 (local address 127.29.112.131:45211)
0811 18:57:16.859103 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:16.859110 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:16.859396 (+ 286us) server_negotiation.cc:407] Connection header received
0811 18:57:16.859580 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:16.859584 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:16.859652 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:16.859742 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:57:16.860267 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:16.860998 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:16.863221 (+ 2223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:16.863975 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:16.864098 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:16.864686 (+ 588us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:57:16.864822 (+ 136us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:57:16.864911 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:16.865116 (+ 205us) server_negotiation.cc:299] Negotiation successful
0811 18:57:16.865209 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":314}
I20260811 18:57:16.865732 1829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:16.858991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.131:45211 (local address 127.29.112.131:59379)
0811 18:57:16.859265 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:16.859292 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:16.859447 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:16.859823 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:16.859828 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:16.859844 (+ 16us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 18:57:16.860108 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:16.860123 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:16.861169 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:16.861176 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:16.862970 (+ 1794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:16.862989 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:16.864167 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:16.864176 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:16.864402 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:16.864461 (+ 59us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 18:57:16.864947 (+ 486us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 18:57:16.864962 (+ 15us) client_negotiation.cc:769] Sending connection context
0811 18:57:16.865072 (+ 110us) client_negotiation.cc:240] Negotiation successful
0811 18:57:16.865268 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65}
I20260811 18:57:16.877498 1126 raft_consensus.cc:1261] T fceef53720a24ab1b44711bab8a69d4b P 937550b3337d417c9d5ec16caa6f3b09 [term 1 FOLLOWER]: Refusing update from remote peer 4c0fb9abd83245b289dc12805882cc9e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:57:16.877511 1370 raft_consensus.cc:1261] T fceef53720a24ab1b44711bab8a69d4b P 482b0d44ef0941dd9f67913d024d29ba [term 1 FOLLOWER]: Refusing update from remote peer 4c0fb9abd83245b289dc12805882cc9e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:57:16.878094 1753 consensus_queue.cc:1035] T fceef53720a24ab1b44711bab8a69d4b P 4c0fb9abd83245b289dc12805882cc9e [LEADER]: Connected to new peer: Peer: permanent_uuid: "482b0d44ef0941dd9f67913d024d29ba" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:16.878443 1831 consensus_queue.cc:1035] T fceef53720a24ab1b44711bab8a69d4b P 4c0fb9abd83245b289dc12805882cc9e [LEADER]: Connected to new peer: Peer: permanent_uuid: "937550b3337d417c9d5ec16caa6f3b09" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 32865 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 18:57:16.905350 1796 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260811 18:57:18.514191 853 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43842:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:57:18.514631 853 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43842:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:57:18.519918 853 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 50c83e7e69974b90a46fe19124750a8d: Sending DeleteTablet for 3 replicas of tablet fceef53720a24ab1b44711bab8a69d4b
I20260811 18:57:18.520958 1586 tablet_service.cc:1514] Processing DeleteTablet for tablet fceef53720a24ab1b44711bab8a69d4b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:18 UTC) from {username='kudu', principal='kudu/127.29.112.190@KRBTEST.COM'} at 127.0.0.1:47630
I20260811 18:57:18.520967 1344 tablet_service.cc:1514] Processing DeleteTablet for tablet fceef53720a24ab1b44711bab8a69d4b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:18 UTC) from {username='kudu', principal='kudu/127.29.112.190@KRBTEST.COM'} at 127.0.0.1:38916
I20260811 18:57:18.521034 1096 tablet_service.cc:1514] Processing DeleteTablet for tablet fceef53720a24ab1b44711bab8a69d4b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:18 UTC) from {username='kudu', principal='kudu/127.29.112.190@KRBTEST.COM'} at 127.0.0.1:57678
I20260811 18:57:18.533319 1888 tablet_replica.cc:330] T fceef53720a24ab1b44711bab8a69d4b P 4c0fb9abd83245b289dc12805882cc9e: stopping tablet replica
I20260811 18:57:18.533702 1892 tablet_replica.cc:330] T fceef53720a24ab1b44711bab8a69d4b P 937550b3337d417c9d5ec16caa6f3b09: stopping tablet replica
I20260811 18:57:18.533952 1888 raft_consensus.cc:2229] T fceef53720a24ab1b44711bab8a69d4b P 4c0fb9abd83245b289dc12805882cc9e [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:18.533987 1892 raft_consensus.cc:2229] T fceef53720a24ab1b44711bab8a69d4b P 937550b3337d417c9d5ec16caa6f3b09 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:18.534283 1892 raft_consensus.cc:2258] T fceef53720a24ab1b44711bab8a69d4b P 937550b3337d417c9d5ec16caa6f3b09 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:18.534256 1890 tablet_replica.cc:330] T fceef53720a24ab1b44711bab8a69d4b P 482b0d44ef0941dd9f67913d024d29ba: stopping tablet replica
I20260811 18:57:18.534608 1890 raft_consensus.cc:2229] T fceef53720a24ab1b44711bab8a69d4b P 482b0d44ef0941dd9f67913d024d29ba [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:18.534688 1888 raft_consensus.cc:2258] T fceef53720a24ab1b44711bab8a69d4b P 4c0fb9abd83245b289dc12805882cc9e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:18.534950 1890 raft_consensus.cc:2258] T fceef53720a24ab1b44711bab8a69d4b P 482b0d44ef0941dd9f67913d024d29ba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:18.536098 1890 ts_tablet_manager.cc:1905] T fceef53720a24ab1b44711bab8a69d4b P 482b0d44ef0941dd9f67913d024d29ba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:18.536113 1892 ts_tablet_manager.cc:1905] T fceef53720a24ab1b44711bab8a69d4b P 937550b3337d417c9d5ec16caa6f3b09: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:18.536113 1888 ts_tablet_manager.cc:1905] T fceef53720a24ab1b44711bab8a69d4b P 4c0fb9abd83245b289dc12805882cc9e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:18.543234 1892 ts_tablet_manager.cc:1918] T fceef53720a24ab1b44711bab8a69d4b P 937550b3337d417c9d5ec16caa6f3b09: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 18:57:18.543213 1888 ts_tablet_manager.cc:1918] T fceef53720a24ab1b44711bab8a69d4b P 4c0fb9abd83245b289dc12805882cc9e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 18:57:18.543377 1888 log.cc:1196] T fceef53720a24ab1b44711bab8a69d4b P 4c0fb9abd83245b289dc12805882cc9e: Deleting WAL directory at /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/minicluster-data/ts-2/wal/wals/fceef53720a24ab1b44711bab8a69d4b
I20260811 18:57:18.543475 1892 log.cc:1196] T fceef53720a24ab1b44711bab8a69d4b P 937550b3337d417c9d5ec16caa6f3b09: Deleting WAL directory at /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/minicluster-data/ts-0/wal/wals/fceef53720a24ab1b44711bab8a69d4b
I20260811 18:57:18.543952 1890 ts_tablet_manager.cc:1918] T fceef53720a24ab1b44711bab8a69d4b P 482b0d44ef0941dd9f67913d024d29ba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 18:57:18.544008 1892 ts_tablet_manager.cc:1939] T fceef53720a24ab1b44711bab8a69d4b P 937550b3337d417c9d5ec16caa6f3b09: Deleting consensus metadata
I20260811 18:57:18.544009 1888 ts_tablet_manager.cc:1939] T fceef53720a24ab1b44711bab8a69d4b P 4c0fb9abd83245b289dc12805882cc9e: Deleting consensus metadata
I20260811 18:57:18.544188 1890 log.cc:1196] T fceef53720a24ab1b44711bab8a69d4b P 482b0d44ef0941dd9f67913d024d29ba: Deleting WAL directory at /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/minicluster-data/ts-1/wal/wals/fceef53720a24ab1b44711bab8a69d4b
I20260811 18:57:18.544687 1890 ts_tablet_manager.cc:1939] T fceef53720a24ab1b44711bab8a69d4b P 482b0d44ef0941dd9f67913d024d29ba: Deleting consensus metadata
I20260811 18:57:18.545255 829 catalog_manager.cc:4841] TS 937550b3337d417c9d5ec16caa6f3b09 (127.29.112.129:32865): tablet fceef53720a24ab1b44711bab8a69d4b (table test-table [id=2d4e5f51a6834c89b7eb1f9cff740dce]) successfully deleted
I20260811 18:57:18.545255 825 catalog_manager.cc:4841] TS 4c0fb9abd83245b289dc12805882cc9e (127.29.112.131:45211): tablet fceef53720a24ab1b44711bab8a69d4b (table test-table [id=2d4e5f51a6834c89b7eb1f9cff740dce]) successfully deleted
I20260811 18:57:18.546101 826 catalog_manager.cc:4841] TS 482b0d44ef0941dd9f67913d024d29ba (127.29.112.130:33619): tablet fceef53720a24ab1b44711bab8a69d4b (table test-table [id=2d4e5f51a6834c89b7eb1f9cff740dce]) successfully deleted
Aug 11 18:57:18 dist-test-slave-h6n0 krb5kdc[771](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474638, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 18:57:18 dist-test-slave-h6n0 krb5kdc[771](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474638, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 18:57:18.600943 1908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:18.577591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43860 (local address 127.29.112.190:39833)
0811 18:57:18.578420 (+ 829us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:18.578426 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:18.578453 (+ 27us) server_negotiation.cc:407] Connection header received
0811 18:57:18.578555 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:18.578600 (+ 45us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:18.578698 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:18.578822 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:18.580648 (+ 1826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:18.581903 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:18.583460 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:18.583949 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:18.584080 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:18.593240 (+ 9160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:18.593246 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:18.593250 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:18.593293 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:18.596908 (+ 3615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:18.597747 (+ 839us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:18.597758 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:18.597761 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:18.598003 (+ 242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:18.598461 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:18.598467 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:18.598472 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:18.599071 (+ 599us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:18.599498 (+ 427us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:18.599841 (+ 343us) server_negotiation.cc:299] Negotiation successful
0811 18:57:18.600163 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":662,"thread_start_us":436,"threads_started":1}
I20260811 18:57:18.606894 853 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43860:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:18.607651 853 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:18.616217 1096 tablet_service.cc:1467] Processing CreateTablet for tablet 46d58090962e41658a400f247bb1354a (DEFAULT_TABLE table=test-table [id=5a72b1a3919e4693b700bf9fabba6936]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:18.616500 1344 tablet_service.cc:1467] Processing CreateTablet for tablet 46d58090962e41658a400f247bb1354a (DEFAULT_TABLE table=test-table [id=5a72b1a3919e4693b700bf9fabba6936]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:18.616992 1096 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46d58090962e41658a400f247bb1354a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:18.616962 1586 tablet_service.cc:1467] Processing CreateTablet for tablet 46d58090962e41658a400f247bb1354a (DEFAULT_TABLE table=test-table [id=5a72b1a3919e4693b700bf9fabba6936]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:18.617066 1344 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46d58090962e41658a400f247bb1354a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:18.617467 1586 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46d58090962e41658a400f247bb1354a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:18.625468 1909 tablet_bootstrap.cc:492] T 46d58090962e41658a400f247bb1354a P 482b0d44ef0941dd9f67913d024d29ba: Bootstrap starting.
I20260811 18:57:18.629145 1910 tablet_bootstrap.cc:492] T 46d58090962e41658a400f247bb1354a P 4c0fb9abd83245b289dc12805882cc9e: Bootstrap starting.
I20260811 18:57:18.629627 1909 tablet_bootstrap.cc:654] T 46d58090962e41658a400f247bb1354a P 482b0d44ef0941dd9f67913d024d29ba: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:18.631023 1912 tablet_bootstrap.cc:492] T 46d58090962e41658a400f247bb1354a P 937550b3337d417c9d5ec16caa6f3b09: Bootstrap starting.
I20260811 18:57:18.634765 1912 tablet_bootstrap.cc:654] T 46d58090962e41658a400f247bb1354a P 937550b3337d417c9d5ec16caa6f3b09: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:18.635854 1910 tablet_bootstrap.cc:654] T 46d58090962e41658a400f247bb1354a P 4c0fb9abd83245b289dc12805882cc9e: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:18.635919 1909 tablet_bootstrap.cc:492] T 46d58090962e41658a400f247bb1354a P 482b0d44ef0941dd9f67913d024d29ba: No bootstrap required, opened a new log
I20260811 18:57:18.636312 1909 ts_tablet_manager.cc:1397] T 46d58090962e41658a400f247bb1354a P 482b0d44ef0941dd9f67913d024d29ba: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.005s
I20260811 18:57:18.637462 1909 raft_consensus.cc:348] T 46d58090962e41658a400f247bb1354a P 482b0d44ef0941dd9f67913d024d29ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937550b3337d417c9d5ec16caa6f3b09" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 32865 } } peers { permanent_uuid: "482b0d44ef0941dd9f67913d024d29ba" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33619 } } peers { permanent_uuid: "4c0fb9abd83245b289dc12805882cc9e" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45211 } }
I20260811 18:57:18.637719 1909 raft_consensus.cc:374] T 46d58090962e41658a400f247bb1354a P 482b0d44ef0941dd9f67913d024d29ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:18.637812 1909 raft_consensus.cc:729] T 46d58090962e41658a400f247bb1354a P 482b0d44ef0941dd9f67913d024d29ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482b0d44ef0941dd9f67913d024d29ba, State: Initialized, Role: FOLLOWER
I20260811 18:57:18.638123 1909 consensus_queue.cc:260] T 46d58090962e41658a400f247bb1354a P 482b0d44ef0941dd9f67913d024d29ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937550b3337d417c9d5ec16caa6f3b09" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 32865 } } peers { permanent_uuid: "482b0d44ef0941dd9f67913d024d29ba" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33619 } } peers { permanent_uuid: "4c0fb9abd83245b289dc12805882cc9e" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45211 } }
I20260811 18:57:18.638983 1912 tablet_bootstrap.cc:492] T 46d58090962e41658a400f247bb1354a P 937550b3337d417c9d5ec16caa6f3b09: No bootstrap required, opened a new log
I20260811 18:57:18.639300 1912 ts_tablet_manager.cc:1397] T 46d58090962e41658a400f247bb1354a P 937550b3337d417c9d5ec16caa6f3b09: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.007s
I20260811 18:57:18.641064 1912 raft_consensus.cc:348] T 46d58090962e41658a400f247bb1354a P 937550b3337d417c9d5ec16caa6f3b09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937550b3337d417c9d5ec16caa6f3b09" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 32865 } } peers { permanent_uuid: "482b0d44ef0941dd9f67913d024d29ba" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33619 } } peers { permanent_uuid: "4c0fb9abd83245b289dc12805882cc9e" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45211 } }
I20260811 18:57:18.641436 1912 raft_consensus.cc:374] T 46d58090962e41658a400f247bb1354a P 937550b3337d417c9d5ec16caa6f3b09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:18.641548 1912 raft_consensus.cc:729] T 46d58090962e41658a400f247bb1354a P 937550b3337d417c9d5ec16caa6f3b09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 937550b3337d417c9d5ec16caa6f3b09, State: Initialized, Role: FOLLOWER
I20260811 18:57:18.641858 1912 consensus_queue.cc:260] T 46d58090962e41658a400f247bb1354a P 937550b3337d417c9d5ec16caa6f3b09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937550b3337d417c9d5ec16caa6f3b09" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 32865 } } peers { permanent_uuid: "482b0d44ef0941dd9f67913d024d29ba" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33619 } } peers { permanent_uuid: "4c0fb9abd83245b289dc12805882cc9e" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45211 } }
I20260811 18:57:18.642150 1909 ts_tablet_manager.cc:1428] T 46d58090962e41658a400f247bb1354a P 482b0d44ef0941dd9f67913d024d29ba: Time spent starting tablet: real 0.006s user 0.005s sys 0.002s
I20260811 18:57:18.643007 1912 ts_tablet_manager.cc:1428] T 46d58090962e41658a400f247bb1354a P 937550b3337d417c9d5ec16caa6f3b09: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260811 18:57:18.643128 1910 tablet_bootstrap.cc:492] T 46d58090962e41658a400f247bb1354a P 4c0fb9abd83245b289dc12805882cc9e: No bootstrap required, opened a new log
I20260811 18:57:18.643333 1910 ts_tablet_manager.cc:1397] T 46d58090962e41658a400f247bb1354a P 4c0fb9abd83245b289dc12805882cc9e: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.007s
I20260811 18:57:18.644308 1910 raft_consensus.cc:348] T 46d58090962e41658a400f247bb1354a P 4c0fb9abd83245b289dc12805882cc9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937550b3337d417c9d5ec16caa6f3b09" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 32865 } } peers { permanent_uuid: "482b0d44ef0941dd9f67913d024d29ba" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33619 } } peers { permanent_uuid: "4c0fb9abd83245b289dc12805882cc9e" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45211 } }
I20260811 18:57:18.644622 1910 raft_consensus.cc:374] T 46d58090962e41658a400f247bb1354a P 4c0fb9abd83245b289dc12805882cc9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:18.644722 1910 raft_consensus.cc:729] T 46d58090962e41658a400f247bb1354a P 4c0fb9abd83245b289dc12805882cc9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c0fb9abd83245b289dc12805882cc9e, State: Initialized, Role: FOLLOWER
I20260811 18:57:18.645013 1910 consensus_queue.cc:260] T 46d58090962e41658a400f247bb1354a P 4c0fb9abd83245b289dc12805882cc9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937550b3337d417c9d5ec16caa6f3b09" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 32865 } } peers { permanent_uuid: "482b0d44ef0941dd9f67913d024d29ba" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33619 } } peers { permanent_uuid: "4c0fb9abd83245b289dc12805882cc9e" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45211 } }
I20260811 18:57:18.645630 1910 ts_tablet_manager.cc:1428] T 46d58090962e41658a400f247bb1354a P 4c0fb9abd83245b289dc12805882cc9e: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:57:18.825321 1920 raft_consensus.cc:482] T 46d58090962e41658a400f247bb1354a P 482b0d44ef0941dd9f67913d024d29ba [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:18.825649 1920 raft_consensus.cc:504] T 46d58090962e41658a400f247bb1354a P 482b0d44ef0941dd9f67913d024d29ba [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937550b3337d417c9d5ec16caa6f3b09" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 32865 } } peers { permanent_uuid: "482b0d44ef0941dd9f67913d024d29ba" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33619 } } peers { permanent_uuid: "4c0fb9abd83245b289dc12805882cc9e" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45211 } }
I20260811 18:57:18.826514 1920 leader_election.cc:288] T 46d58090962e41658a400f247bb1354a P 482b0d44ef0941dd9f67913d024d29ba [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 937550b3337d417c9d5ec16caa6f3b09 (127.29.112.129:32865), 4c0fb9abd83245b289dc12805882cc9e (127.29.112.131:45211)
I20260811 18:57:18.827183 1636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46d58090962e41658a400f247bb1354a" candidate_uuid: "482b0d44ef0941dd9f67913d024d29ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c0fb9abd83245b289dc12805882cc9e" is_pre_election: true
I20260811 18:57:18.827203 1125 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46d58090962e41658a400f247bb1354a" candidate_uuid: "482b0d44ef0941dd9f67913d024d29ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "937550b3337d417c9d5ec16caa6f3b09" is_pre_election: true
I20260811 18:57:18.827473 1636 raft_consensus.cc:2454] T 46d58090962e41658a400f247bb1354a P 4c0fb9abd83245b289dc12805882cc9e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 482b0d44ef0941dd9f67913d024d29ba in term 0.
I20260811 18:57:18.827860 1125 raft_consensus.cc:2454] T 46d58090962e41658a400f247bb1354a P 937550b3337d417c9d5ec16caa6f3b09 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 482b0d44ef0941dd9f67913d024d29ba in term 0.
I20260811 18:57:18.828048 1248 leader_election.cc:302] T 46d58090962e41658a400f247bb1354a P 482b0d44ef0941dd9f67913d024d29ba [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 482b0d44ef0941dd9f67913d024d29ba, 4c0fb9abd83245b289dc12805882cc9e; no voters:
I20260811 18:57:18.828687 1920 raft_consensus.cc:2781] T 46d58090962e41658a400f247bb1354a P 482b0d44ef0941dd9f67913d024d29ba [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:18.828836 1920 raft_consensus.cc:482] T 46d58090962e41658a400f247bb1354a P 482b0d44ef0941dd9f67913d024d29ba [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:18.829043 1920 raft_consensus.cc:3037] T 46d58090962e41658a400f247bb1354a P 482b0d44ef0941dd9f67913d024d29ba [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:18.831893 1920 raft_consensus.cc:504] T 46d58090962e41658a400f247bb1354a P 482b0d44ef0941dd9f67913d024d29ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937550b3337d417c9d5ec16caa6f3b09" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 32865 } } peers { permanent_uuid: "482b0d44ef0941dd9f67913d024d29ba" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33619 } } peers { permanent_uuid: "4c0fb9abd83245b289dc12805882cc9e" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45211 } }
I20260811 18:57:18.833437 1920 leader_election.cc:288] T 46d58090962e41658a400f247bb1354a P 482b0d44ef0941dd9f67913d024d29ba [CANDIDATE]: Term 1 election: Requested vote from peers 937550b3337d417c9d5ec16caa6f3b09 (127.29.112.129:32865), 4c0fb9abd83245b289dc12805882cc9e (127.29.112.131:45211)
I20260811 18:57:18.834311 1125 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46d58090962e41658a400f247bb1354a" candidate_uuid: "482b0d44ef0941dd9f67913d024d29ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "937550b3337d417c9d5ec16caa6f3b09"
I20260811 18:57:18.834311 1636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46d58090962e41658a400f247bb1354a" candidate_uuid: "482b0d44ef0941dd9f67913d024d29ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c0fb9abd83245b289dc12805882cc9e"
I20260811 18:57:18.834617 1636 raft_consensus.cc:3037] T 46d58090962e41658a400f247bb1354a P 4c0fb9abd83245b289dc12805882cc9e [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:18.834617 1125 raft_consensus.cc:3037] T 46d58090962e41658a400f247bb1354a P 937550b3337d417c9d5ec16caa6f3b09 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:18.838205 1636 raft_consensus.cc:2454] T 46d58090962e41658a400f247bb1354a P 4c0fb9abd83245b289dc12805882cc9e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 482b0d44ef0941dd9f67913d024d29ba in term 1.
I20260811 18:57:18.838208 1125 raft_consensus.cc:2454] T 46d58090962e41658a400f247bb1354a P 937550b3337d417c9d5ec16caa6f3b09 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 482b0d44ef0941dd9f67913d024d29ba in term 1.
I20260811 18:57:18.838812 1248 leader_election.cc:302] T 46d58090962e41658a400f247bb1354a P 482b0d44ef0941dd9f67913d024d29ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 482b0d44ef0941dd9f67913d024d29ba, 4c0fb9abd83245b289dc12805882cc9e; no voters:
I20260811 18:57:18.839175 1920 raft_consensus.cc:2781] T 46d58090962e41658a400f247bb1354a P 482b0d44ef0941dd9f67913d024d29ba [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:18.839723 1920 raft_consensus.cc:686] T 46d58090962e41658a400f247bb1354a P 482b0d44ef0941dd9f67913d024d29ba [term 1 LEADER]: Becoming Leader. State: Replica: 482b0d44ef0941dd9f67913d024d29ba, State: Running, Role: LEADER
I20260811 18:57:18.840062 1920 consensus_queue.cc:237] T 46d58090962e41658a400f247bb1354a P 482b0d44ef0941dd9f67913d024d29ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937550b3337d417c9d5ec16caa6f3b09" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 32865 } } peers { permanent_uuid: "482b0d44ef0941dd9f67913d024d29ba" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33619 } } peers { permanent_uuid: "4c0fb9abd83245b289dc12805882cc9e" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45211 } }
I20260811 18:57:18.843536 849 catalog_manager.cc:5495] T 46d58090962e41658a400f247bb1354a P 482b0d44ef0941dd9f67913d024d29ba reported cstate change: term changed from 0 to 1, leader changed from <none> to 482b0d44ef0941dd9f67913d024d29ba (127.29.112.130). New cstate: current_term: 1 leader_uuid: "482b0d44ef0941dd9f67913d024d29ba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937550b3337d417c9d5ec16caa6f3b09" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 32865 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "482b0d44ef0941dd9f67913d024d29ba" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4c0fb9abd83245b289dc12805882cc9e" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45211 } health_report { overall_health: UNKNOWN } } }
I20260811 18:57:18.889649 1930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:18.883289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38930 (local address 127.29.112.130:33619)
0811 18:57:18.884088 (+ 799us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:18.884093 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:18.884115 (+ 22us) server_negotiation.cc:407] Connection header received
0811 18:57:18.884176 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:18.884180 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:18.884250 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:18.884403 (+ 153us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:57:18.884936 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:18.885789 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:18.887154 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:18.887798 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:18.887941 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:18.888764 (+ 823us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:57:18.888885 (+ 121us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:57:18.889022 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:18.889184 (+ 162us) server_negotiation.cc:299] Negotiation successful
0811 18:57:18.889266 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":668,"thread_start_us":468,"threads_started":1}
I20260811 18:57:18.901993 1125 raft_consensus.cc:1261] T 46d58090962e41658a400f247bb1354a P 937550b3337d417c9d5ec16caa6f3b09 [term 1 FOLLOWER]: Refusing update from remote peer 482b0d44ef0941dd9f67913d024d29ba: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:57:18.902050 1636 raft_consensus.cc:1261] T 46d58090962e41658a400f247bb1354a P 4c0fb9abd83245b289dc12805882cc9e [term 1 FOLLOWER]: Refusing update from remote peer 482b0d44ef0941dd9f67913d024d29ba: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:57:18.902683 1922 consensus_queue.cc:1035] T 46d58090962e41658a400f247bb1354a P 482b0d44ef0941dd9f67913d024d29ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "937550b3337d417c9d5ec16caa6f3b09" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 32865 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:18.903182 1920 consensus_queue.cc:1035] T 46d58090962e41658a400f247bb1354a P 482b0d44ef0941dd9f67913d024d29ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c0fb9abd83245b289dc12805882cc9e" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45211 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:18.961694 849 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43860:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:57:18.962023 849 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43860:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:57:18.965731 849 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 50c83e7e69974b90a46fe19124750a8d: Sending DeleteTablet for 3 replicas of tablet 46d58090962e41658a400f247bb1354a
I20260811 18:57:18.967000 1096 tablet_service.cc:1514] Processing DeleteTablet for tablet 46d58090962e41658a400f247bb1354a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:18 UTC) from {username='kudu', principal='kudu/127.29.112.190@KRBTEST.COM'} at 127.0.0.1:57678
I20260811 18:57:18.967140 1344 tablet_service.cc:1514] Processing DeleteTablet for tablet 46d58090962e41658a400f247bb1354a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:18 UTC) from {username='kudu', principal='kudu/127.29.112.190@KRBTEST.COM'} at 127.0.0.1:38916
I20260811 18:57:18.967535 1890 tablet_replica.cc:330] T 46d58090962e41658a400f247bb1354a P 482b0d44ef0941dd9f67913d024d29ba: stopping tablet replica
I20260811 18:57:18.967504 1586 tablet_service.cc:1514] Processing DeleteTablet for tablet 46d58090962e41658a400f247bb1354a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:18 UTC) from {username='kudu', principal='kudu/127.29.112.190@KRBTEST.COM'} at 127.0.0.1:47630
I20260811 18:57:18.967805 1888 tablet_replica.cc:330] T 46d58090962e41658a400f247bb1354a P 4c0fb9abd83245b289dc12805882cc9e: stopping tablet replica
I20260811 18:57:18.967798 1890 raft_consensus.cc:2229] T 46d58090962e41658a400f247bb1354a P 482b0d44ef0941dd9f67913d024d29ba [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:18.967962 1892 tablet_replica.cc:330] T 46d58090962e41658a400f247bb1354a P 937550b3337d417c9d5ec16caa6f3b09: stopping tablet replica
I20260811 18:57:18.968053 1888 raft_consensus.cc:2229] T 46d58090962e41658a400f247bb1354a P 4c0fb9abd83245b289dc12805882cc9e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:18.968215 1890 raft_consensus.cc:2258] T 46d58090962e41658a400f247bb1354a P 482b0d44ef0941dd9f67913d024d29ba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:18.968310 1888 raft_consensus.cc:2258] T 46d58090962e41658a400f247bb1354a P 4c0fb9abd83245b289dc12805882cc9e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:18.968333 1892 raft_consensus.cc:2229] T 46d58090962e41658a400f247bb1354a P 937550b3337d417c9d5ec16caa6f3b09 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:18.968605 1892 raft_consensus.cc:2258] T 46d58090962e41658a400f247bb1354a P 937550b3337d417c9d5ec16caa6f3b09 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:18.969627 1888 ts_tablet_manager.cc:1905] T 46d58090962e41658a400f247bb1354a P 4c0fb9abd83245b289dc12805882cc9e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:18.969722 1890 ts_tablet_manager.cc:1905] T 46d58090962e41658a400f247bb1354a P 482b0d44ef0941dd9f67913d024d29ba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:18.970446 849 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43860:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:57:18.970849 1892 ts_tablet_manager.cc:1905] T 46d58090962e41658a400f247bb1354a P 937550b3337d417c9d5ec16caa6f3b09: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:57:18.971623 849 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:18.978580 1890 ts_tablet_manager.cc:1918] T 46d58090962e41658a400f247bb1354a P 482b0d44ef0941dd9f67913d024d29ba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 18:57:18.978785 1890 log.cc:1196] T 46d58090962e41658a400f247bb1354a P 482b0d44ef0941dd9f67913d024d29ba: Deleting WAL directory at /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/minicluster-data/ts-1/wal/wals/46d58090962e41658a400f247bb1354a
I20260811 18:57:18.979295 1890 ts_tablet_manager.cc:1939] T 46d58090962e41658a400f247bb1354a P 482b0d44ef0941dd9f67913d024d29ba: Deleting consensus metadata
I20260811 18:57:18.980444 1892 ts_tablet_manager.cc:1918] T 46d58090962e41658a400f247bb1354a P 937550b3337d417c9d5ec16caa6f3b09: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 18:57:18.980660 1892 log.cc:1196] T 46d58090962e41658a400f247bb1354a P 937550b3337d417c9d5ec16caa6f3b09: Deleting WAL directory at /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/minicluster-data/ts-0/wal/wals/46d58090962e41658a400f247bb1354a
I20260811 18:57:18.980688 826 catalog_manager.cc:4841] TS 482b0d44ef0941dd9f67913d024d29ba (127.29.112.130:33619): tablet 46d58090962e41658a400f247bb1354a (table test-table [id=5a72b1a3919e4693b700bf9fabba6936]) successfully deleted
I20260811 18:57:18.981254 1892 ts_tablet_manager.cc:1939] T 46d58090962e41658a400f247bb1354a P 937550b3337d417c9d5ec16caa6f3b09: Deleting consensus metadata
I20260811 18:57:18.981631 1096 tablet_service.cc:1467] Processing CreateTablet for tablet 14eee2caaf01495f805231adf8b04904 (DEFAULT_TABLE table=test-table [id=0db873c9d20447868c54cddaed21821b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:18.981814 1344 tablet_service.cc:1467] Processing CreateTablet for tablet 14eee2caaf01495f805231adf8b04904 (DEFAULT_TABLE table=test-table [id=0db873c9d20447868c54cddaed21821b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:18.982156 1096 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14eee2caaf01495f805231adf8b04904. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:18.982774 1344 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14eee2caaf01495f805231adf8b04904. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:18.982818 829 catalog_manager.cc:4841] TS 937550b3337d417c9d5ec16caa6f3b09 (127.29.112.129:32865): tablet 46d58090962e41658a400f247bb1354a (table test-table [id=5a72b1a3919e4693b700bf9fabba6936]) successfully deleted
I20260811 18:57:18.982038 1586 tablet_service.cc:1467] Processing CreateTablet for tablet 14eee2caaf01495f805231adf8b04904 (DEFAULT_TABLE table=test-table [id=0db873c9d20447868c54cddaed21821b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:18.983527 1586 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14eee2caaf01495f805231adf8b04904. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:18.987367 1888 ts_tablet_manager.cc:1918] T 46d58090962e41658a400f247bb1354a P 4c0fb9abd83245b289dc12805882cc9e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 18:57:18.987660 1888 log.cc:1196] T 46d58090962e41658a400f247bb1354a P 4c0fb9abd83245b289dc12805882cc9e: Deleting WAL directory at /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/minicluster-data/ts-2/wal/wals/46d58090962e41658a400f247bb1354a
I20260811 18:57:18.988481 1888 ts_tablet_manager.cc:1939] T 46d58090962e41658a400f247bb1354a P 4c0fb9abd83245b289dc12805882cc9e: Deleting consensus metadata
I20260811 18:57:18.990149 1912 tablet_bootstrap.cc:492] T 14eee2caaf01495f805231adf8b04904 P 937550b3337d417c9d5ec16caa6f3b09: Bootstrap starting.
I20260811 18:57:18.990862 825 catalog_manager.cc:4841] TS 4c0fb9abd83245b289dc12805882cc9e (127.29.112.131:45211): tablet 46d58090962e41658a400f247bb1354a (table test-table [id=5a72b1a3919e4693b700bf9fabba6936]) successfully deleted
I20260811 18:57:18.992516 1912 tablet_bootstrap.cc:654] T 14eee2caaf01495f805231adf8b04904 P 937550b3337d417c9d5ec16caa6f3b09: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:18.992565 1909 tablet_bootstrap.cc:492] T 14eee2caaf01495f805231adf8b04904 P 482b0d44ef0941dd9f67913d024d29ba: Bootstrap starting.
I20260811 18:57:18.993678 1910 tablet_bootstrap.cc:492] T 14eee2caaf01495f805231adf8b04904 P 4c0fb9abd83245b289dc12805882cc9e: Bootstrap starting.
I20260811 18:57:18.995851 1909 tablet_bootstrap.cc:654] T 14eee2caaf01495f805231adf8b04904 P 482b0d44ef0941dd9f67913d024d29ba: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:18.995932 1910 tablet_bootstrap.cc:654] T 14eee2caaf01495f805231adf8b04904 P 4c0fb9abd83245b289dc12805882cc9e: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:18.997440 1912 tablet_bootstrap.cc:492] T 14eee2caaf01495f805231adf8b04904 P 937550b3337d417c9d5ec16caa6f3b09: No bootstrap required, opened a new log
I20260811 18:57:18.997788 1912 ts_tablet_manager.cc:1397] T 14eee2caaf01495f805231adf8b04904 P 937550b3337d417c9d5ec16caa6f3b09: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.004s
I20260811 18:57:18.999006 1910 tablet_bootstrap.cc:492] T 14eee2caaf01495f805231adf8b04904 P 4c0fb9abd83245b289dc12805882cc9e: No bootstrap required, opened a new log
I20260811 18:57:18.999169 1912 raft_consensus.cc:348] T 14eee2caaf01495f805231adf8b04904 P 937550b3337d417c9d5ec16caa6f3b09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937550b3337d417c9d5ec16caa6f3b09" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 32865 } } peers { permanent_uuid: "482b0d44ef0941dd9f67913d024d29ba" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33619 } } peers { permanent_uuid: "4c0fb9abd83245b289dc12805882cc9e" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45211 } }
I20260811 18:57:18.999506 1912 raft_consensus.cc:374] T 14eee2caaf01495f805231adf8b04904 P 937550b3337d417c9d5ec16caa6f3b09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:18.999619 1912 raft_consensus.cc:729] T 14eee2caaf01495f805231adf8b04904 P 937550b3337d417c9d5ec16caa6f3b09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 937550b3337d417c9d5ec16caa6f3b09, State: Initialized, Role: FOLLOWER
I20260811 18:57:18.999627 1910 ts_tablet_manager.cc:1397] T 14eee2caaf01495f805231adf8b04904 P 4c0fb9abd83245b289dc12805882cc9e: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 18:57:19.000057 1912 consensus_queue.cc:260] T 14eee2caaf01495f805231adf8b04904 P 937550b3337d417c9d5ec16caa6f3b09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937550b3337d417c9d5ec16caa6f3b09" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 32865 } } peers { permanent_uuid: "482b0d44ef0941dd9f67913d024d29ba" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33619 } } peers { permanent_uuid: "4c0fb9abd83245b289dc12805882cc9e" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45211 } }
I20260811 18:57:19.000382 1909 tablet_bootstrap.cc:492] T 14eee2caaf01495f805231adf8b04904 P 482b0d44ef0941dd9f67913d024d29ba: No bootstrap required, opened a new log
I20260811 18:57:19.000566 1909 ts_tablet_manager.cc:1397] T 14eee2caaf01495f805231adf8b04904 P 482b0d44ef0941dd9f67913d024d29ba: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260811 18:57:19.000835 1912 ts_tablet_manager.cc:1428] T 14eee2caaf01495f805231adf8b04904 P 937550b3337d417c9d5ec16caa6f3b09: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 18:57:19.000792 1910 raft_consensus.cc:348] T 14eee2caaf01495f805231adf8b04904 P 4c0fb9abd83245b289dc12805882cc9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937550b3337d417c9d5ec16caa6f3b09" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 32865 } } peers { permanent_uuid: "482b0d44ef0941dd9f67913d024d29ba" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33619 } } peers { permanent_uuid: "4c0fb9abd83245b289dc12805882cc9e" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45211 } }
I20260811 18:57:19.001048 1910 raft_consensus.cc:374] T 14eee2caaf01495f805231adf8b04904 P 4c0fb9abd83245b289dc12805882cc9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:19.001153 1910 raft_consensus.cc:729] T 14eee2caaf01495f805231adf8b04904 P 4c0fb9abd83245b289dc12805882cc9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c0fb9abd83245b289dc12805882cc9e, State: Initialized, Role: FOLLOWER
I20260811 18:57:19.001451 1910 consensus_queue.cc:260] T 14eee2caaf01495f805231adf8b04904 P 4c0fb9abd83245b289dc12805882cc9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937550b3337d417c9d5ec16caa6f3b09" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 32865 } } peers { permanent_uuid: "482b0d44ef0941dd9f67913d024d29ba" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33619 } } peers { permanent_uuid: "4c0fb9abd83245b289dc12805882cc9e" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45211 } }
I20260811 18:57:19.001385 1909 raft_consensus.cc:348] T 14eee2caaf01495f805231adf8b04904 P 482b0d44ef0941dd9f67913d024d29ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937550b3337d417c9d5ec16caa6f3b09" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 32865 } } peers { permanent_uuid: "482b0d44ef0941dd9f67913d024d29ba" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33619 } } peers { permanent_uuid: "4c0fb9abd83245b289dc12805882cc9e" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45211 } }
I20260811 18:57:19.001683 1909 raft_consensus.cc:374] T 14eee2caaf01495f805231adf8b04904 P 482b0d44ef0941dd9f67913d024d29ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:19.001925 1910 ts_tablet_manager.cc:1428] T 14eee2caaf01495f805231adf8b04904 P 4c0fb9abd83245b289dc12805882cc9e: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260811 18:57:19.001763 1909 raft_consensus.cc:729] T 14eee2caaf01495f805231adf8b04904 P 482b0d44ef0941dd9f67913d024d29ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482b0d44ef0941dd9f67913d024d29ba, State: Initialized, Role: FOLLOWER
I20260811 18:57:19.002341 1909 consensus_queue.cc:260] T 14eee2caaf01495f805231adf8b04904 P 482b0d44ef0941dd9f67913d024d29ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937550b3337d417c9d5ec16caa6f3b09" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 32865 } } peers { permanent_uuid: "482b0d44ef0941dd9f67913d024d29ba" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33619 } } peers { permanent_uuid: "4c0fb9abd83245b289dc12805882cc9e" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45211 } }
I20260811 18:57:19.003167 1909 ts_tablet_manager.cc:1428] T 14eee2caaf01495f805231adf8b04904 P 482b0d44ef0941dd9f67913d024d29ba: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:57:19.056918 1920 raft_consensus.cc:482] T 14eee2caaf01495f805231adf8b04904 P 482b0d44ef0941dd9f67913d024d29ba [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:19.057219 1920 raft_consensus.cc:504] T 14eee2caaf01495f805231adf8b04904 P 482b0d44ef0941dd9f67913d024d29ba [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937550b3337d417c9d5ec16caa6f3b09" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 32865 } } peers { permanent_uuid: "482b0d44ef0941dd9f67913d024d29ba" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33619 } } peers { permanent_uuid: "4c0fb9abd83245b289dc12805882cc9e" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45211 } }
I20260811 18:57:19.057961 1920 leader_election.cc:288] T 14eee2caaf01495f805231adf8b04904 P 482b0d44ef0941dd9f67913d024d29ba [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 937550b3337d417c9d5ec16caa6f3b09 (127.29.112.129:32865), 4c0fb9abd83245b289dc12805882cc9e (127.29.112.131:45211)
I20260811 18:57:19.058712 1126 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14eee2caaf01495f805231adf8b04904" candidate_uuid: "482b0d44ef0941dd9f67913d024d29ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "937550b3337d417c9d5ec16caa6f3b09" is_pre_election: true
I20260811 18:57:19.058779 1636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14eee2caaf01495f805231adf8b04904" candidate_uuid: "482b0d44ef0941dd9f67913d024d29ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c0fb9abd83245b289dc12805882cc9e" is_pre_election: true
I20260811 18:57:19.058976 1126 raft_consensus.cc:2454] T 14eee2caaf01495f805231adf8b04904 P 937550b3337d417c9d5ec16caa6f3b09 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 482b0d44ef0941dd9f67913d024d29ba in term 0.
I20260811 18:57:19.059149 1636 raft_consensus.cc:2454] T 14eee2caaf01495f805231adf8b04904 P 4c0fb9abd83245b289dc12805882cc9e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 482b0d44ef0941dd9f67913d024d29ba in term 0.
I20260811 18:57:19.059500 1258 leader_election.cc:302] T 14eee2caaf01495f805231adf8b04904 P 482b0d44ef0941dd9f67913d024d29ba [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 482b0d44ef0941dd9f67913d024d29ba, 937550b3337d417c9d5ec16caa6f3b09; no voters:
I20260811 18:57:19.059800 1920 raft_consensus.cc:2781] T 14eee2caaf01495f805231adf8b04904 P 482b0d44ef0941dd9f67913d024d29ba [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:19.059921 1920 raft_consensus.cc:482] T 14eee2caaf01495f805231adf8b04904 P 482b0d44ef0941dd9f67913d024d29ba [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:19.060077 1920 raft_consensus.cc:3037] T 14eee2caaf01495f805231adf8b04904 P 482b0d44ef0941dd9f67913d024d29ba [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:19.062409 1920 raft_consensus.cc:504] T 14eee2caaf01495f805231adf8b04904 P 482b0d44ef0941dd9f67913d024d29ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937550b3337d417c9d5ec16caa6f3b09" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 32865 } } peers { permanent_uuid: "482b0d44ef0941dd9f67913d024d29ba" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33619 } } peers { permanent_uuid: "4c0fb9abd83245b289dc12805882cc9e" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45211 } }
I20260811 18:57:19.063143 1920 leader_election.cc:288] T 14eee2caaf01495f805231adf8b04904 P 482b0d44ef0941dd9f67913d024d29ba [CANDIDATE]: Term 1 election: Requested vote from peers 937550b3337d417c9d5ec16caa6f3b09 (127.29.112.129:32865), 4c0fb9abd83245b289dc12805882cc9e (127.29.112.131:45211)
I20260811 18:57:19.063710 1126 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14eee2caaf01495f805231adf8b04904" candidate_uuid: "482b0d44ef0941dd9f67913d024d29ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "937550b3337d417c9d5ec16caa6f3b09"
I20260811 18:57:19.063759 1636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14eee2caaf01495f805231adf8b04904" candidate_uuid: "482b0d44ef0941dd9f67913d024d29ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c0fb9abd83245b289dc12805882cc9e"
I20260811 18:57:19.063939 1126 raft_consensus.cc:3037] T 14eee2caaf01495f805231adf8b04904 P 937550b3337d417c9d5ec16caa6f3b09 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:19.064030 1636 raft_consensus.cc:3037] T 14eee2caaf01495f805231adf8b04904 P 4c0fb9abd83245b289dc12805882cc9e [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:19.066751 1636 raft_consensus.cc:2454] T 14eee2caaf01495f805231adf8b04904 P 4c0fb9abd83245b289dc12805882cc9e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 482b0d44ef0941dd9f67913d024d29ba in term 1.
I20260811 18:57:19.067356 1248 leader_election.cc:302] T 14eee2caaf01495f805231adf8b04904 P 482b0d44ef0941dd9f67913d024d29ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 482b0d44ef0941dd9f67913d024d29ba, 4c0fb9abd83245b289dc12805882cc9e; no voters:
I20260811 18:57:19.067801 1920 raft_consensus.cc:2781] T 14eee2caaf01495f805231adf8b04904 P 482b0d44ef0941dd9f67913d024d29ba [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:19.068291 1920 raft_consensus.cc:686] T 14eee2caaf01495f805231adf8b04904 P 482b0d44ef0941dd9f67913d024d29ba [term 1 LEADER]: Becoming Leader. State: Replica: 482b0d44ef0941dd9f67913d024d29ba, State: Running, Role: LEADER
I20260811 18:57:19.068037 1126 raft_consensus.cc:2454] T 14eee2caaf01495f805231adf8b04904 P 937550b3337d417c9d5ec16caa6f3b09 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 482b0d44ef0941dd9f67913d024d29ba in term 1.
I20260811 18:57:19.068679 1920 consensus_queue.cc:237] T 14eee2caaf01495f805231adf8b04904 P 482b0d44ef0941dd9f67913d024d29ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937550b3337d417c9d5ec16caa6f3b09" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 32865 } } peers { permanent_uuid: "482b0d44ef0941dd9f67913d024d29ba" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33619 } } peers { permanent_uuid: "4c0fb9abd83245b289dc12805882cc9e" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45211 } }
I20260811 18:57:19.072842 849 catalog_manager.cc:5495] T 14eee2caaf01495f805231adf8b04904 P 482b0d44ef0941dd9f67913d024d29ba reported cstate change: term changed from 0 to 1, leader changed from <none> to 482b0d44ef0941dd9f67913d024d29ba (127.29.112.130). New cstate: current_term: 1 leader_uuid: "482b0d44ef0941dd9f67913d024d29ba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937550b3337d417c9d5ec16caa6f3b09" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 32865 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "482b0d44ef0941dd9f67913d024d29ba" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4c0fb9abd83245b289dc12805882cc9e" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45211 } health_report { overall_health: UNKNOWN } } }
I20260811 18:57:19.114248 1126 raft_consensus.cc:1261] T 14eee2caaf01495f805231adf8b04904 P 937550b3337d417c9d5ec16caa6f3b09 [term 1 FOLLOWER]: Refusing update from remote peer 482b0d44ef0941dd9f67913d024d29ba: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:57:19.114595 1636 raft_consensus.cc:1261] T 14eee2caaf01495f805231adf8b04904 P 4c0fb9abd83245b289dc12805882cc9e [term 1 FOLLOWER]: Refusing update from remote peer 482b0d44ef0941dd9f67913d024d29ba: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:57:19.115003 1920 consensus_queue.cc:1035] T 14eee2caaf01495f805231adf8b04904 P 482b0d44ef0941dd9f67913d024d29ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "937550b3337d417c9d5ec16caa6f3b09" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 32865 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:19.115242 1922 consensus_queue.cc:1035] T 14eee2caaf01495f805231adf8b04904 P 482b0d44ef0941dd9f67913d024d29ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c0fb9abd83245b289dc12805882cc9e" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 45211 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 18:57:19.141366 1857 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260811 18:57:20.780617 849 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43860:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:57:20.781431 849 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43860:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:57:20.788959 849 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 50c83e7e69974b90a46fe19124750a8d: Sending DeleteTablet for 3 replicas of tablet 14eee2caaf01495f805231adf8b04904
I20260811 18:57:20.790236 1096 tablet_service.cc:1514] Processing DeleteTablet for tablet 14eee2caaf01495f805231adf8b04904 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:20 UTC) from {username='kudu', principal='kudu/127.29.112.190@KRBTEST.COM'} at 127.0.0.1:57678
I20260811 18:57:20.790853 1344 tablet_service.cc:1514] Processing DeleteTablet for tablet 14eee2caaf01495f805231adf8b04904 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:20 UTC) from {username='kudu', principal='kudu/127.29.112.190@KRBTEST.COM'} at 127.0.0.1:38916
I20260811 18:57:20.791419 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 975
I20260811 18:57:20.791599 1586 tablet_service.cc:1514] Processing DeleteTablet for tablet 14eee2caaf01495f805231adf8b04904 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:20 UTC) from {username='kudu', principal='kudu/127.29.112.190@KRBTEST.COM'} at 127.0.0.1:47630
I20260811 18:57:20.791745 1991 tablet_replica.cc:330] T 14eee2caaf01495f805231adf8b04904 P 937550b3337d417c9d5ec16caa6f3b09: stopping tablet replica
I20260811 18:57:20.792097 1991 raft_consensus.cc:2229] T 14eee2caaf01495f805231adf8b04904 P 937550b3337d417c9d5ec16caa6f3b09 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:20.792469 1993 tablet_replica.cc:330] T 14eee2caaf01495f805231adf8b04904 P 4c0fb9abd83245b289dc12805882cc9e: stopping tablet replica
I20260811 18:57:20.792546 1991 raft_consensus.cc:2258] T 14eee2caaf01495f805231adf8b04904 P 937550b3337d417c9d5ec16caa6f3b09 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:20.792771 1993 raft_consensus.cc:2229] T 14eee2caaf01495f805231adf8b04904 P 4c0fb9abd83245b289dc12805882cc9e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:20.793041 1993 raft_consensus.cc:2258] T 14eee2caaf01495f805231adf8b04904 P 4c0fb9abd83245b289dc12805882cc9e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:20.796340 1991 ts_tablet_manager.cc:1905] T 14eee2caaf01495f805231adf8b04904 P 937550b3337d417c9d5ec16caa6f3b09: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:20.796602 1993 ts_tablet_manager.cc:1905] T 14eee2caaf01495f805231adf8b04904 P 4c0fb9abd83245b289dc12805882cc9e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:20.801299 1992 tablet_replica.cc:330] T 14eee2caaf01495f805231adf8b04904 P 482b0d44ef0941dd9f67913d024d29ba: stopping tablet replica
I20260811 18:57:20.801769 1992 raft_consensus.cc:2229] T 14eee2caaf01495f805231adf8b04904 P 482b0d44ef0941dd9f67913d024d29ba [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:20.802619 1992 raft_consensus.cc:2258] T 14eee2caaf01495f805231adf8b04904 P 482b0d44ef0941dd9f67913d024d29ba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:20.803905 1992 ts_tablet_manager.cc:1905] T 14eee2caaf01495f805231adf8b04904 P 482b0d44ef0941dd9f67913d024d29ba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:20.807706 1993 ts_tablet_manager.cc:1918] T 14eee2caaf01495f805231adf8b04904 P 4c0fb9abd83245b289dc12805882cc9e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 18:57:20.808168 1993 log.cc:1196] T 14eee2caaf01495f805231adf8b04904 P 4c0fb9abd83245b289dc12805882cc9e: Deleting WAL directory at /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/minicluster-data/ts-2/wal/wals/14eee2caaf01495f805231adf8b04904
I20260811 18:57:20.808979 1993 ts_tablet_manager.cc:1939] T 14eee2caaf01495f805231adf8b04904 P 4c0fb9abd83245b289dc12805882cc9e: Deleting consensus metadata
I20260811 18:57:20.810564 825 catalog_manager.cc:4841] TS 4c0fb9abd83245b289dc12805882cc9e (127.29.112.131:45211): tablet 14eee2caaf01495f805231adf8b04904 (table test-table [id=0db873c9d20447868c54cddaed21821b]) successfully deleted
Aug 11 18:57:20 dist-test-slave-h6n0 krb5kdc[771](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474638, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.29.112.129@KRBTEST.COM
I20260811 18:57:20.814922 1992 ts_tablet_manager.cc:1918] T 14eee2caaf01495f805231adf8b04904 P 482b0d44ef0941dd9f67913d024d29ba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 18:57:20.815380 1992 log.cc:1196] T 14eee2caaf01495f805231adf8b04904 P 482b0d44ef0941dd9f67913d024d29ba: Deleting WAL directory at /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/minicluster-data/ts-1/wal/wals/14eee2caaf01495f805231adf8b04904
I20260811 18:57:20.816391 1992 ts_tablet_manager.cc:1939] T 14eee2caaf01495f805231adf8b04904 P 482b0d44ef0941dd9f67913d024d29ba: Deleting consensus metadata
I20260811 18:57:20.804899 1991 ts_tablet_manager.cc:1918] T 14eee2caaf01495f805231adf8b04904 P 937550b3337d417c9d5ec16caa6f3b09: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 18:57:20.817353 1991 log.cc:1196] T 14eee2caaf01495f805231adf8b04904 P 937550b3337d417c9d5ec16caa6f3b09: Deleting WAL directory at /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474615205622-30146-0/minicluster-data/ts-0/wal/wals/14eee2caaf01495f805231adf8b04904
I20260811 18:57:20.817894 1991 ts_tablet_manager.cc:1939] T 14eee2caaf01495f805231adf8b04904 P 937550b3337d417c9d5ec16caa6f3b09: Deleting consensus metadata
I20260811 18:57:20.818346 826 catalog_manager.cc:4841] TS 482b0d44ef0941dd9f67913d024d29ba (127.29.112.130:33619): tablet 14eee2caaf01495f805231adf8b04904 (table test-table [id=0db873c9d20447868c54cddaed21821b]) successfully deleted
I20260811 18:57:20.819458 829 catalog_manager.cc:4841] TS 937550b3337d417c9d5ec16caa6f3b09 (127.29.112.129:32865): tablet 14eee2caaf01495f805231adf8b04904 (table test-table [id=0db873c9d20447868c54cddaed21821b]) successfully deleted
I20260811 18:57:20.831110 1995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:20.793306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42850 (local address 127.29.112.129:32865)
0811 18:57:20.796189 (+ 2883us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:20.796200 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:20.796236 (+ 36us) server_negotiation.cc:407] Connection header received
0811 18:57:20.796326 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:20.796336 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:20.796475 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:20.796627 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:20.799942 (+ 3315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:20.801552 (+ 1610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:20.803915 (+ 2363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:20.804897 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:20.805037 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:20.821205 (+ 16168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:20.821238 (+ 33us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:20.821264 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:20.821384 (+ 120us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:20.827140 (+ 5756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:20.828386 (+ 1246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:20.828396 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:20.828400 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:20.828558 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:20.829092 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:20.829097 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:20.829100 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:20.829556 (+ 456us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:20.829889 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:20.830180 (+ 291us) server_negotiation.cc:299] Negotiation successful
0811 18:57:20.830457 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2669,"thread_start_us":2540,"threads_started":1}
W20260811 18:57:20.832193 1201 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42850
W20260811 18:57:20.833256 30146 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 975: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 18:57:20.833516 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 975
I20260811 18:57:20.872542 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 1213
Aug 11 18:57:20 dist-test-slave-h6n0 krb5kdc[771](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474638, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.29.112.130@KRBTEST.COM
I20260811 18:57:20.895684 1996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:20.873782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56752 (local address 127.29.112.130:33619)
0811 18:57:20.875254 (+ 1472us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:20.875260 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:20.875284 (+ 24us) server_negotiation.cc:407] Connection header received
0811 18:57:20.875361 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:20.875365 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:20.875482 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:20.875631 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:20.877567 (+ 1936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:20.878979 (+ 1412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:20.880628 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:20.881405 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:20.881525 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:20.886903 (+ 5378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:20.886921 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:20.886938 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:20.886997 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:20.891730 (+ 4733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:20.892705 (+ 975us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:20.892714 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:20.892717 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:20.892912 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:20.893584 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:20.893588 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:20.893591 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:20.894104 (+ 513us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:20.894411 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:20.895058 (+ 647us) server_negotiation.cc:299] Negotiation successful
0811 18:57:20.895348 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1104,"thread_start_us":864,"threads_started":1}
W20260811 18:57:20.896898 1455 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56752
W20260811 18:57:20.897779 30146 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 1213: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 18:57:20.898098 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 1213
I20260811 18:57:20.932946 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 1474
Aug 11 18:57:20 dist-test-slave-h6n0 krb5kdc[771](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474638, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.29.112.131@KRBTEST.COM
I20260811 18:57:20.951221 1999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:20.934029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42168 (local address 127.29.112.131:45211)
0811 18:57:20.934700 (+ 671us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:20.934705 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:20.934722 (+ 17us) server_negotiation.cc:407] Connection header received
0811 18:57:20.934781 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:20.934785 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:20.934858 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:20.934964 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:20.936541 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:20.937439 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:20.938751 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:20.939203 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:20.939307 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:20.944059 (+ 4752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:20.944073 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:20.944089 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:20.944125 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:20.947498 (+ 3373us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:20.948252 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:20.948259 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:20.948262 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:20.948415 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:20.948962 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:20.948970 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:20.948972 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:20.949569 (+ 597us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:20.949955 (+ 386us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:20.950445 (+ 490us) server_negotiation.cc:299] Negotiation successful
0811 18:57:20.950826 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":483,"thread_start_us":305,"threads_started":1}
W20260811 18:57:20.952076 1702 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42168
W20260811 18:57:20.952976 30146 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 1474: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 18:57:20.953204 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 1474
I20260811 18:57:20.994778 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 794
I20260811 18:57:20.996170 928 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:57:21.138700 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 794
2026-08-11T18:57:21Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (10852 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474615205622-30146-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:57:21 dist-test-slave-h6n0 krb5kdc[2007](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:57:21 dist-test-slave-h6n0 krb5kdc[2007](info): set up 2 sockets
Aug 11 18:57:21 dist-test-slave-h6n0 krb5kdc[2007](info): commencing operation
krb5kdc: starting...
W20260811 18:57:22.564880 30146 mini_kdc.cc:121] Time spent starting KDC: real 1.364s user 0.000s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474615205622-30146-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474615205622-30146-0/krb5kdc/test-user.keytab.
Aug 11 18:57:22 dist-test-slave-h6n0 krb5kdc[2007](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474642, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:57:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:57:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.112.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab.
Entry for principal kudu/127.29.112.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab.
I20260811 18:57:22.808262 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474615205622-30146-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474615205622-30146-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474615205622-30146-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.190:40863
--webserver_interface=127.29.112.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab
--principal=kudu/127.29.112.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:35257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.190:40863
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:57:23.166159 2031 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:57:23.166522 2031 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:57:23.166749 2031 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:57:23.175587 2031 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:57:23.175776 2031 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:57:23.175982 2031 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:57:23.176087 2031 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:57:23.176136 2031 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:57:23.188236 2031 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:35257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474615205622-30146-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474615205622-30146-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.112.190:40863
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab
--principal=kudu/127.29.112.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.190:40863
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.112.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474615205622-30146-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:57:23.189657 2031 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:57:23.191730 2031 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:57:23.207662 2045 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:23.207595 2036 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:23.207672 2037 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:57:23.208766 2031 server_base.cc:1034] running on GCE node
I20260811 18:57:23.209663 2031 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:57:23.211319 2031 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:57:23.212589 2031 hybrid_clock.cc:648] HybridClock initialized: now 1786474643212543 us; error 73 us; skew 500 ppm
I20260811 18:57:23.212973 2031 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:57:23 dist-test-slave-h6n0 krb5kdc[2007](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474643, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:57:23.216756 2031 init.cc:377] Logged in from keytab as kudu/127.29.112.190@KRBTEST.COM (short username kudu)
I20260811 18:57:23.219312 2031 webserver.cc:460] Webserver started at http://127.29.112.190:36795/ using document root <none> and password file <none>
I20260811 18:57:23.220072 2031 fs_manager.cc:359] Metadata directory not provided
I20260811 18:57:23.220228 2031 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:57:23.220588 2031 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:57:23.223938 2031 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474615205622-30146-0/minicluster-data/master-0/data/instance:
uuid: "8cd9639b44f8433d93d0343d65173809"
format_stamp: "Formatted at 2026-08-11 18:57:23 on dist-test-slave-h6n0"
server_key: "9ee50d9c0c73695790aa2a811599b585"
server_key_iv: "c20dc1ec3cf55bbd769f5a071765ba23"
server_key_version: "encryptionkey@0"
I20260811 18:57:23.224803 2031 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474615205622-30146-0/minicluster-data/master-0/wal/instance:
uuid: "8cd9639b44f8433d93d0343d65173809"
format_stamp: "Formatted at 2026-08-11 18:57:23 on dist-test-slave-h6n0"
server_key: "9ee50d9c0c73695790aa2a811599b585"
server_key_iv: "c20dc1ec3cf55bbd769f5a071765ba23"
server_key_version: "encryptionkey@0"
I20260811 18:57:23.231182 2031 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260811 18:57:23.235158 2058 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:23.237178 2031 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 18:57:23.237510 2031 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474615205622-30146-0/minicluster-data/master-0/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474615205622-30146-0/minicluster-data/master-0/wal
uuid: "8cd9639b44f8433d93d0343d65173809"
format_stamp: "Formatted at 2026-08-11 18:57:23 on dist-test-slave-h6n0"
server_key: "9ee50d9c0c73695790aa2a811599b585"
server_key_iv: "c20dc1ec3cf55bbd769f5a071765ba23"
server_key_version: "encryptionkey@0"
I20260811 18:57:23.237809 2031 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474615205622-30146-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474615205622-30146-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474615205622-30146-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:57:23.259438 2031 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:57:23.264365 2031 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:57:23.264708 2031 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:57:23.285526 2031 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.190:40863
I20260811 18:57:23.285552 2149 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.190:40863 every 8 connection(s)
I20260811 18:57:23.287038 2031 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
I20260811 18:57:23.290987 2150 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:23.295048 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 2031
I20260811 18:57:23.295281 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474615205622-30146-0/minicluster-data/master-0/wal/instance
I20260811 18:57:23.296204 30146 external_mini_cluster.cc:1428] Setting key b4cf27b62659437dba8000ab3fb39faf
I20260811 18:57:23.303894 2150 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8cd9639b44f8433d93d0343d65173809: Bootstrap starting.
I20260811 18:57:23.307430 2150 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8cd9639b44f8433d93d0343d65173809: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:23.309155 2150 log.cc:824] T 00000000000000000000000000000000 P 8cd9639b44f8433d93d0343d65173809: Log is configured to *not* fsync() on all Append() calls
I20260811 18:57:23.312628 2150 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8cd9639b44f8433d93d0343d65173809: No bootstrap required, opened a new log
Aug 11 18:57:23 dist-test-slave-h6n0 krb5kdc[2007](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474642, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 18:57:23.324052 2153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:23.298834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57290 (local address 127.29.112.190:40863)
0811 18:57:23.299920 (+ 1086us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:23.299932 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:23.299975 (+ 43us) server_negotiation.cc:407] Connection header received
0811 18:57:23.302371 (+ 2396us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:23.302395 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:23.302922 (+ 527us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:23.303321 (+ 399us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:23.305020 (+ 1699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:23.306385 (+ 1365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:23.307748 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:23.309170 (+ 1422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:23.309374 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:23.314258 (+ 4884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:23.314270 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:23.314290 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:23.314346 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:23.319044 (+ 4698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:23.319824 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:23.319834 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:23.319845 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:23.320080 (+ 235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:23.320476 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:23.320482 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:23.320486 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:23.321263 (+ 777us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:23.321615 (+ 352us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:23.321962 (+ 347us) server_negotiation.cc:299] Negotiation successful
0811 18:57:23.322477 (+ 515us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":875,"thread_start_us":584,"threads_started":1}
I20260811 18:57:23.325187 2150 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8cd9639b44f8433d93d0343d65173809 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cd9639b44f8433d93d0343d65173809" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 40863 } }
I20260811 18:57:23.325700 2150 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8cd9639b44f8433d93d0343d65173809 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:23.325845 2150 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8cd9639b44f8433d93d0343d65173809 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8cd9639b44f8433d93d0343d65173809, State: Initialized, Role: FOLLOWER
I20260811 18:57:23.326597 2150 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8cd9639b44f8433d93d0343d65173809 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cd9639b44f8433d93d0343d65173809" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 40863 } }
I20260811 18:57:23.326893 2150 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8cd9639b44f8433d93d0343d65173809 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:57:23.327054 2150 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8cd9639b44f8433d93d0343d65173809 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:57:23.327268 2150 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8cd9639b44f8433d93d0343d65173809 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:23.329712 2150 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8cd9639b44f8433d93d0343d65173809 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cd9639b44f8433d93d0343d65173809" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 40863 } }
I20260811 18:57:23.330327 2150 leader_election.cc:302] T 00000000000000000000000000000000 P 8cd9639b44f8433d93d0343d65173809 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8cd9639b44f8433d93d0343d65173809; no voters:
I20260811 18:57:23.331030 2150 leader_election.cc:288] T 00000000000000000000000000000000 P 8cd9639b44f8433d93d0343d65173809 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:57:23.331403 2158 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8cd9639b44f8433d93d0343d65173809 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:23.332244 2158 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8cd9639b44f8433d93d0343d65173809 [term 1 LEADER]: Becoming Leader. State: Replica: 8cd9639b44f8433d93d0343d65173809, State: Running, Role: LEADER
I20260811 18:57:23.332793 2158 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8cd9639b44f8433d93d0343d65173809 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cd9639b44f8433d93d0343d65173809" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 40863 } }
I20260811 18:57:23.333344 2150 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8cd9639b44f8433d93d0343d65173809 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:57:23.336144 2160 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8cd9639b44f8433d93d0343d65173809 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8cd9639b44f8433d93d0343d65173809" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cd9639b44f8433d93d0343d65173809" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 40863 } } }
I20260811 18:57:23.336122 2163 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8cd9639b44f8433d93d0343d65173809 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8cd9639b44f8433d93d0343d65173809. Latest consensus state: current_term: 1 leader_uuid: "8cd9639b44f8433d93d0343d65173809" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cd9639b44f8433d93d0343d65173809" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 40863 } } }
I20260811 18:57:23.336654 2160 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8cd9639b44f8433d93d0343d65173809 [sys.catalog]: This master's current role is: LEADER
I20260811 18:57:23.336679 2163 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8cd9639b44f8433d93d0343d65173809 [sys.catalog]: This master's current role is: LEADER
I20260811 18:57:23.338522 2167 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:57:23.345024 2167 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:57:23.355082 2167 catalog_manager.cc:1347] Generated new cluster ID: a1837cdbae004b20a477e4fa78120bfc
I20260811 18:57:23.355199 2167 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:57:23.373817 2167 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:57:23.375411 2167 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:57:23.383698 2167 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8cd9639b44f8433d93d0343d65173809: Generated new TSK 0
I20260811 18:57:23.385020 2167 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.112.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.129.keytab.
Entry for principal kudu/127.29.112.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.129.keytab.
I20260811 18:57:23.450793 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.129:0
--local_ip_for_outbound_sockets=127.29.112.129
--webserver_interface=127.29.112.129
--webserver_port=0
--tserver_master_addrs=127.29.112.190:40863
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.129.keytab
--principal=kudu/127.29.112.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:35257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:57:23.794050 2192 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:57:23.794432 2192 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:57:23.794642 2192 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:57:23.802461 2192 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:57:23.802642 2192 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:57:23.802817 2192 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.129
I20260811 18:57:23.813606 2192 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:35257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.129.keytab
--principal=kudu/127.29.112.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.112.129
--webserver_port=0
--tserver_master_addrs=127.29.112.190:40863
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.129
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:57:23.815157 2192 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:57:23.816556 2192 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:57:23.834831 2197 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:23.834748 2201 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:23.834910 2203 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:57:23.835676 2192 server_base.cc:1034] running on GCE node
I20260811 18:57:23.836405 2192 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:57:23.837636 2192 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:57:23.838882 2192 hybrid_clock.cc:648] HybridClock initialized: now 1786474643838835 us; error 33 us; skew 500 ppm
I20260811 18:57:23.839246 2192 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:57:23 dist-test-slave-h6n0 krb5kdc[2007](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474643, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:57:23.846820 2192 init.cc:377] Logged in from keytab as kudu/127.29.112.129@KRBTEST.COM (short username kudu)
I20260811 18:57:23.850229 2192 webserver.cc:460] Webserver started at http://127.29.112.129:36697/ using document root <none> and password file <none>
I20260811 18:57:23.851090 2192 fs_manager.cc:359] Metadata directory not provided
I20260811 18:57:23.851213 2192 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:57:23.851487 2192 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:57:23.854801 2192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-0/data/instance:
uuid: "1feddc3e486f4da6a50ed89c5c01f9e8"
format_stamp: "Formatted at 2026-08-11 18:57:23 on dist-test-slave-h6n0"
server_key: "3ea3ddce14c6ba6dd57e49365d2065e0"
server_key_iv: "09d47dfec292ef329256361460e1a9db"
server_key_version: "encryptionkey@0"
I20260811 18:57:23.855592 2192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-0/wal/instance:
uuid: "1feddc3e486f4da6a50ed89c5c01f9e8"
format_stamp: "Formatted at 2026-08-11 18:57:23 on dist-test-slave-h6n0"
server_key: "3ea3ddce14c6ba6dd57e49365d2065e0"
server_key_iv: "09d47dfec292ef329256361460e1a9db"
server_key_version: "encryptionkey@0"
I20260811 18:57:23.861541 2192 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.004s
I20260811 18:57:23.866256 2218 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:23.867892 2192 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 18:57:23.868088 2192 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-0/wal
uuid: "1feddc3e486f4da6a50ed89c5c01f9e8"
format_stamp: "Formatted at 2026-08-11 18:57:23 on dist-test-slave-h6n0"
server_key: "3ea3ddce14c6ba6dd57e49365d2065e0"
server_key_iv: "09d47dfec292ef329256361460e1a9db"
server_key_version: "encryptionkey@0"
I20260811 18:57:23.868301 2192 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:57:23.890038 2192 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:57:23.896601 2192 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:57:23.897011 2192 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:57:23.898090 2192 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:57:23.899760 2192 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:57:23.899983 2192 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:23.900173 2192 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:57:23.900243 2192 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:23.939424 2192 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.129:38605
I20260811 18:57:23.939431 2427 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.129:38605 every 8 connection(s)
I20260811 18:57:23.940999 2192 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-0/data/info.pb
I20260811 18:57:23.945657 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 2192
I20260811 18:57:23.945896 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-0/wal/instance
I20260811 18:57:23.946653 30146 external_mini_cluster.cc:1428] Setting key 1489f7e43eec9047ff54631c770a4fca
Aug 11 18:57:23 dist-test-slave-h6n0 krb5kdc[2007](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474643, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.129@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 18:57:23.963717 2432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:23.944106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.129:46227 (local address 127.29.112.190:40863)
0811 18:57:23.944621 (+ 515us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:23.944628 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:23.945779 (+ 1151us) server_negotiation.cc:407] Connection header received
0811 18:57:23.948022 (+ 2243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:23.948032 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:23.948112 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:23.948214 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:23.950049 (+ 1835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:23.951299 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:23.952862 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:23.953607 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:23.953759 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:23.958004 (+ 4245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:23.958010 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:23.958013 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:23.958045 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:23.960537 (+ 2492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:23.961298 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:23.961303 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:23.961307 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:23.961438 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:23.961890 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:23.961896 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:23.961899 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:23.962171 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:23.962331 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:23.963109 (+ 778us) server_negotiation.cc:299] Negotiation successful
0811 18:57:23.963339 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":242,"threads_started":1}
I20260811 18:57:23.964720 2433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:23.944700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:40863 (local address 127.29.112.129:46227)
0811 18:57:23.945577 (+ 877us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:23.945629 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:23.947710 (+ 2081us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:23.948502 (+ 792us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:23.948515 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:23.949219 (+ 704us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:57:23.949772 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:23.949789 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:23.951472 (+ 1683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:23.951476 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:23.952709 (+ 1233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:23.952721 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:23.953874 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:23.953880 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:23.954106 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:23.954876 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:57:23.954903 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:57:23.957773 (+ 2870us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:57:23.960693 (+ 2920us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:23.960702 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:23.960718 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:23.961160 (+ 442us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:23.961557 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:23.961564 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:23.961568 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:23.961768 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:23.962308 (+ 540us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:57:23.962317 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:57:23.962732 (+ 415us) client_negotiation.cc:769] Sending connection context
0811 18:57:23.963014 (+ 282us) client_negotiation.cc:240] Negotiation successful
0811 18:57:23.963572 (+ 558us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":618,"thread_start_us":392,"threads_started":1}
I20260811 18:57:23.966588 2428 heartbeater.cc:347] Connected to a master server at 127.29.112.190:40863
I20260811 18:57:23.966913 2428 heartbeater.cc:464] Registering TS with master...
I20260811 18:57:23.967648 2428 heartbeater.cc:511] Master 127.29.112.190:40863 requested a full tablet report, sending...
I20260811 18:57:23.970060 2089 ts_manager.cc:194] Registered new tserver with Master: 1feddc3e486f4da6a50ed89c5c01f9e8 (127.29.112.129:38605)
WARNING: no policy specified for kudu/127.29.112.130@KRBTEST.COM; defaulting to no policy
I20260811 18:57:23.971807 2089 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.129@KRBTEST.COM'} at 127.29.112.129:46227
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.130.keytab.
Entry for principal kudu/127.29.112.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.130.keytab.
I20260811 18:57:24.002920 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.130:0
--local_ip_for_outbound_sockets=127.29.112.130
--webserver_interface=127.29.112.130
--webserver_port=0
--tserver_master_addrs=127.29.112.190:40863
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.130.keytab
--principal=kudu/127.29.112.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:35257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:57:24.355655 2441 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:57:24.355924 2441 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:57:24.356083 2441 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:57:24.363488 2441 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:57:24.363595 2441 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:57:24.363788 2441 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.130
I20260811 18:57:24.373687 2441 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:35257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.130.keytab
--principal=kudu/127.29.112.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.112.130
--webserver_port=0
--tserver_master_addrs=127.29.112.190:40863
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.130
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:57:24.374986 2441 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:57:24.376296 2441 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:57:24.389572 2452 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:24.389608 2454 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:24.389645 2461 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:57:24.390854 2441 server_base.cc:1034] running on GCE node
I20260811 18:57:24.391481 2441 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:57:24.392424 2441 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:57:24.393651 2441 hybrid_clock.cc:648] HybridClock initialized: now 1786474644393598 us; error 50 us; skew 500 ppm
I20260811 18:57:24.393992 2441 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:57:24 dist-test-slave-h6n0 krb5kdc[2007](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474644, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:57:24.397276 2441 init.cc:377] Logged in from keytab as kudu/127.29.112.130@KRBTEST.COM (short username kudu)
I20260811 18:57:24.398837 2441 webserver.cc:460] Webserver started at http://127.29.112.130:36643/ using document root <none> and password file <none>
I20260811 18:57:24.399530 2441 fs_manager.cc:359] Metadata directory not provided
I20260811 18:57:24.399681 2441 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:57:24.399997 2441 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:57:24.403177 2441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-1/data/instance:
uuid: "eec3ccb8575748178f7d79a5466ef5dd"
format_stamp: "Formatted at 2026-08-11 18:57:24 on dist-test-slave-h6n0"
server_key: "a3f7dd2c9b3a3246fd970566ef00b206"
server_key_iv: "b79c8aa51be3c71f99c7336b29fea831"
server_key_version: "encryptionkey@0"
I20260811 18:57:24.403995 2441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-1/wal/instance:
uuid: "eec3ccb8575748178f7d79a5466ef5dd"
format_stamp: "Formatted at 2026-08-11 18:57:24 on dist-test-slave-h6n0"
server_key: "a3f7dd2c9b3a3246fd970566ef00b206"
server_key_iv: "b79c8aa51be3c71f99c7336b29fea831"
server_key_version: "encryptionkey@0"
I20260811 18:57:24.410060 2441 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.003s
I20260811 18:57:24.413858 2473 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:24.415242 2441 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 18:57:24.415434 2441 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-1/wal
uuid: "eec3ccb8575748178f7d79a5466ef5dd"
format_stamp: "Formatted at 2026-08-11 18:57:24 on dist-test-slave-h6n0"
server_key: "a3f7dd2c9b3a3246fd970566ef00b206"
server_key_iv: "b79c8aa51be3c71f99c7336b29fea831"
server_key_version: "encryptionkey@0"
I20260811 18:57:24.415657 2441 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:57:24.430899 2441 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:57:24.435184 2441 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:57:24.435500 2441 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:57:24.436499 2441 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:57:24.437914 2441 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:57:24.438022 2441 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:24.438174 2441 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:57:24.438252 2441 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:24.476511 2441 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.130:34069
I20260811 18:57:24.476562 2665 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.130:34069 every 8 connection(s)
I20260811 18:57:24.477945 2441 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-1/data/info.pb
I20260811 18:57:24.480144 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 2441
I20260811 18:57:24.480341 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-1/wal/instance
I20260811 18:57:24.480974 30146 external_mini_cluster.cc:1428] Setting key 89ddf706b110186cd7bd2f4cc52a982c
Aug 11 18:57:24 dist-test-slave-h6n0 krb5kdc[2007](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474644, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.130@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 18:57:24.504878 2670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:24.481468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.130:38943 (local address 127.29.112.190:40863)
0811 18:57:24.482139 (+ 671us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:24.482147 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:24.483238 (+ 1091us) server_negotiation.cc:407] Connection header received
0811 18:57:24.487050 (+ 3812us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:24.487056 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:24.487137 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:24.487240 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:24.489631 (+ 2391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:24.490392 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:24.493261 (+ 2869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:24.493826 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:24.494092 (+ 266us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:24.498215 (+ 4123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:24.498220 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:24.498224 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:24.498264 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:24.501477 (+ 3213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:24.502317 (+ 840us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:24.502321 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:24.502323 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:24.502458 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:24.502969 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:24.502973 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:24.502976 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:24.503272 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:24.503440 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:24.504210 (+ 770us) server_negotiation.cc:299] Negotiation successful
0811 18:57:24.504466 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":502,"thread_start_us":363,"threads_started":1}
I20260811 18:57:24.505905 2671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:24.481970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:40863 (local address 127.29.112.130:38943)
0811 18:57:24.482998 (+ 1028us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:24.483077 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:24.486415 (+ 3338us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:24.487570 (+ 1155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:24.487585 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:24.488510 (+ 925us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:57:24.489391 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:24.489420 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:24.490688 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:24.490697 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:24.492997 (+ 2300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:24.493017 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:24.494223 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:24.494229 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:24.494457 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:24.495602 (+ 1145us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:57:24.495627 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:57:24.497942 (+ 2315us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:57:24.501670 (+ 3728us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:24.501680 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:24.501697 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:24.502163 (+ 466us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:24.502611 (+ 448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:24.502617 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:24.502620 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:24.502830 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:24.503412 (+ 582us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:57:24.503421 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:57:24.503809 (+ 388us) client_negotiation.cc:769] Sending connection context
0811 18:57:24.504099 (+ 290us) client_negotiation.cc:240] Negotiation successful
0811 18:57:24.504675 (+ 576us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":695,"thread_start_us":379,"threads_started":1}
I20260811 18:57:24.507723 2666 heartbeater.cc:347] Connected to a master server at 127.29.112.190:40863
I20260811 18:57:24.508107 2666 heartbeater.cc:464] Registering TS with master...
I20260811 18:57:24.508922 2666 heartbeater.cc:511] Master 127.29.112.190:40863 requested a full tablet report, sending...
I20260811 18:57:24.510706 2089 ts_manager.cc:194] Registered new tserver with Master: eec3ccb8575748178f7d79a5466ef5dd (127.29.112.130:34069)
WARNING: no policy specified for kudu/127.29.112.131@KRBTEST.COM; defaulting to no policy
I20260811 18:57:24.512010 2089 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.130@KRBTEST.COM'} at 127.29.112.130:38943
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.131.keytab.
Entry for principal kudu/127.29.112.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.131.keytab.
I20260811 18:57:24.547780 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.131:0
--local_ip_for_outbound_sockets=127.29.112.131
--webserver_interface=127.29.112.131
--webserver_port=0
--tserver_master_addrs=127.29.112.190:40863
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.131.keytab
--principal=kudu/127.29.112.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:35257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:57:24.913764 2676 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:57:24.914072 2676 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:57:24.914250 2676 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:57:24.921756 2676 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:57:24.921877 2676 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:57:24.922043 2676 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.131
I20260811 18:57:24.932852 2676 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:35257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.131.keytab
--principal=kudu/127.29.112.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.112.131
--webserver_port=0
--tserver_master_addrs=127.29.112.190:40863
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.131
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:57:24.934389 2676 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:57:24.935899 2676 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:57:24.952430 2683 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:24.952366 2686 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:24.952327 2689 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:57:24.953497 2676 server_base.cc:1034] running on GCE node
I20260811 18:57:24.954278 2676 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:57:24.955552 2676 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:57:24.957000 2676 hybrid_clock.cc:648] HybridClock initialized: now 1786474644956922 us; error 51 us; skew 500 ppm
I20260811 18:57:24.957657 2676 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:57:24 dist-test-slave-h6n0 krb5kdc[2007](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474644, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:57:24.961795 2676 init.cc:377] Logged in from keytab as kudu/127.29.112.131@KRBTEST.COM (short username kudu)
I20260811 18:57:24.963697 2676 webserver.cc:460] Webserver started at http://127.29.112.131:35319/ using document root <none> and password file <none>
I20260811 18:57:24.964493 2676 fs_manager.cc:359] Metadata directory not provided
I20260811 18:57:24.964633 2676 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:57:24.964994 2676 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:57:24.968159 2676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-2/data/instance:
uuid: "1d655e9be23240eca115bdea7873be39"
format_stamp: "Formatted at 2026-08-11 18:57:24 on dist-test-slave-h6n0"
server_key: "e117329adec4ee5dd410e3443c89f0d0"
server_key_iv: "bac897d8cae55af0869fee14c2a69482"
server_key_version: "encryptionkey@0"
I20260811 18:57:24.969012 2676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-2/wal/instance:
uuid: "1d655e9be23240eca115bdea7873be39"
format_stamp: "Formatted at 2026-08-11 18:57:24 on dist-test-slave-h6n0"
server_key: "e117329adec4ee5dd410e3443c89f0d0"
server_key_iv: "bac897d8cae55af0869fee14c2a69482"
server_key_version: "encryptionkey@0"
I20260811 18:57:24.975394 2428 heartbeater.cc:503] Master 127.29.112.190:40863 was elected leader, sending a full tablet report...
I20260811 18:57:24.975719 2676 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260811 18:57:24.980259 2698 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:24.981871 2676 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.003s
I20260811 18:57:24.982169 2676 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-2/wal
uuid: "1d655e9be23240eca115bdea7873be39"
format_stamp: "Formatted at 2026-08-11 18:57:24 on dist-test-slave-h6n0"
server_key: "e117329adec4ee5dd410e3443c89f0d0"
server_key_iv: "bac897d8cae55af0869fee14c2a69482"
server_key_version: "encryptionkey@0"
I20260811 18:57:24.982420 2676 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:57:24.993799 2676 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:57:24.998420 2676 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:57:24.998835 2676 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:57:25.000391 2676 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:57:25.002061 2676 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:57:25.002140 2676 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:25.002266 2676 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:57:25.002365 2676 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:25.047722 2676 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.131:40357
I20260811 18:57:25.047763 2897 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.131:40357 every 8 connection(s)
I20260811 18:57:25.049095 2676 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-2/data/info.pb
I20260811 18:57:25.053694 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 2676
I20260811 18:57:25.053879 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-2/wal/instance
I20260811 18:57:25.054484 30146 external_mini_cluster.cc:1428] Setting key cb3d18b0f4eec477fe3ac96e16a3dafa
Aug 11 18:57:25 dist-test-slave-h6n0 krb5kdc[2007](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474644, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.131@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 18:57:25.073712 2904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:25.052255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.131:60169 (local address 127.29.112.190:40863)
0811 18:57:25.052806 (+ 551us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:25.052811 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:25.053693 (+ 882us) server_negotiation.cc:407] Connection header received
0811 18:57:25.056391 (+ 2698us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:25.056399 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:25.056499 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:25.056607 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:25.058602 (+ 1995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:25.059567 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:25.061217 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:25.061687 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:25.061805 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:25.065820 (+ 4015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:25.065826 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:25.065830 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:25.065883 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:25.069868 (+ 3985us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:25.070792 (+ 924us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:25.070805 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:25.070808 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:25.070960 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:25.071495 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:25.071503 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:25.071511 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:25.071814 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:25.072013 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:25.072923 (+ 910us) server_negotiation.cc:299] Negotiation successful
0811 18:57:25.073178 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":443,"thread_start_us":304,"threads_started":1}
I20260811 18:57:25.074725 2906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:25.052663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:40863 (local address 127.29.112.131:60169)
0811 18:57:25.053478 (+ 815us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:25.053540 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:25.055832 (+ 2292us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:25.056936 (+ 1104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:25.056946 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:25.057613 (+ 667us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:57:25.058317 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:25.058336 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:25.059758 (+ 1422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:25.059762 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:25.061044 (+ 1282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:25.061067 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:25.061946 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:25.061953 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:25.062188 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:25.063034 (+ 846us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:57:25.063057 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:57:25.065509 (+ 2452us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:57:25.070047 (+ 4538us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:25.070063 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:25.070077 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:25.070653 (+ 576us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:25.071099 (+ 446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:25.071105 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:25.071109 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:25.071354 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:25.071972 (+ 618us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:57:25.071981 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:57:25.072452 (+ 471us) client_negotiation.cc:769] Sending connection context
0811 18:57:25.072804 (+ 352us) client_negotiation.cc:240] Negotiation successful
0811 18:57:25.073396 (+ 592us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524,"thread_start_us":295,"threads_started":1}
I20260811 18:57:25.076187 2898 heartbeater.cc:347] Connected to a master server at 127.29.112.190:40863
I20260811 18:57:25.076525 2898 heartbeater.cc:464] Registering TS with master...
I20260811 18:57:25.077276 2898 heartbeater.cc:511] Master 127.29.112.190:40863 requested a full tablet report, sending...
I20260811 18:57:25.078891 2089 ts_manager.cc:194] Registered new tserver with Master: 1d655e9be23240eca115bdea7873be39 (127.29.112.131:40357)
I20260811 18:57:25.079710 2089 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.131@KRBTEST.COM'} at 127.29.112.131:60169
I20260811 18:57:25.084479 30146 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 18:57:25.125540 2904 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:57:25.123009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57298 (local address 127.29.112.190:40863)
0811 18:57:25.123230 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:25.123235 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:25.123673 (+ 438us) server_negotiation.cc:407] Connection header received
0811 18:57:25.123814 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:25.123818 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:25.123910 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:25.124018 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:25.125347 (+ 1329us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57298: BlockingRecv error: recv got EOF from 127.0.0.1:57298 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
I20260811 18:57:25.128866 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 2192
W20260811 18:57:25.131906 30146 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 2192: Not authorized: Client connection negotiation failed: client connection to 127.29.112.129:38605: server requires authentication, but client does not have Kerberos credentials available
I20260811 18:57:25.132123 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 2192
I20260811 18:57:25.169672 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 2441
W20260811 18:57:25.173172 30146 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 2441: Not authorized: Client connection negotiation failed: client connection to 127.29.112.130:34069: server requires authentication, but client does not have Kerberos credentials available
I20260811 18:57:25.173345 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 2441
I20260811 18:57:25.203084 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 2676
W20260811 18:57:25.207369 30146 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 2676: Not authorized: Client connection negotiation failed: client connection to 127.29.112.131:40357: server requires authentication, but client does not have Kerberos credentials available
I20260811 18:57:25.207588 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 2676
I20260811 18:57:25.235121 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 2031
I20260811 18:57:25.236603 2143 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:57:25.348737 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 2031
2026-08-11T18:57:25Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (4195 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474615205622-30146-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:57:25 dist-test-slave-h6n0 krb5kdc[2928](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:57:25 dist-test-slave-h6n0 krb5kdc[2928](info): set up 2 sockets
Aug 11 18:57:25 dist-test-slave-h6n0 krb5kdc[2928](info): commencing operation
krb5kdc: starting...
W20260811 18:57:26.741586 30146 mini_kdc.cc:121] Time spent starting KDC: real 1.344s user 0.000s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474615205622-30146-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474615205622-30146-0/krb5kdc/test-user.keytab.
Aug 11 18:57:26 dist-test-slave-h6n0 krb5kdc[2928](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474646, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:57:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:57:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.112.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab.
Entry for principal kudu/127.29.112.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab.
I20260811 18:57:26.994928 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474615205622-30146-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474615205622-30146-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474615205622-30146-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.190:39197
--webserver_interface=127.29.112.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab
--principal=kudu/127.29.112.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:34317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.190:39197
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:57:27.343488 2952 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:57:27.343873 2952 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:57:27.344054 2952 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:57:27.351480 2952 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:57:27.351652 2952 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:57:27.351769 2952 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:57:27.351883 2952 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:57:27.351961 2952 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:57:27.362309 2952 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:34317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474615205622-30146-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474615205622-30146-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.112.190:39197
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab
--principal=kudu/127.29.112.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.190:39197
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.112.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474615205622-30146-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:57:27.363813 2952 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:57:27.365187 2952 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:57:27.378036 2960 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:27.377964 2961 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:27.378077 2963 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:57:27.379002 2952 server_base.cc:1034] running on GCE node
I20260811 18:57:27.379907 2952 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:57:27.381449 2952 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:57:27.382647 2952 hybrid_clock.cc:648] HybridClock initialized: now 1786474647382607 us; error 52 us; skew 500 ppm
I20260811 18:57:27.383036 2952 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:57:27 dist-test-slave-h6n0 krb5kdc[2928](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474647, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:57:27.386662 2952 init.cc:377] Logged in from keytab as kudu/127.29.112.190@KRBTEST.COM (short username kudu)
I20260811 18:57:27.388273 2952 webserver.cc:460] Webserver started at http://127.29.112.190:33581/ using document root <none> and password file <none>
I20260811 18:57:27.389091 2952 fs_manager.cc:359] Metadata directory not provided
I20260811 18:57:27.389249 2952 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:57:27.389586 2952 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:57:27.392854 2952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474615205622-30146-0/minicluster-data/master-0/data/instance:
uuid: "626ee287629d41f0a618efa008133984"
format_stamp: "Formatted at 2026-08-11 18:57:27 on dist-test-slave-h6n0"
server_key: "41ce5816a4869487411a8cd1e0923900"
server_key_iv: "cf26aa669b3dbe04907a6b4e21f43bfc"
server_key_version: "encryptionkey@0"
I20260811 18:57:27.393703 2952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474615205622-30146-0/minicluster-data/master-0/wal/instance:
uuid: "626ee287629d41f0a618efa008133984"
format_stamp: "Formatted at 2026-08-11 18:57:27 on dist-test-slave-h6n0"
server_key: "41ce5816a4869487411a8cd1e0923900"
server_key_iv: "cf26aa669b3dbe04907a6b4e21f43bfc"
server_key_version: "encryptionkey@0"
I20260811 18:57:27.399309 2952 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 18:57:27.403631 2975 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:27.405035 2952 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 18:57:27.405247 2952 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474615205622-30146-0/minicluster-data/master-0/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474615205622-30146-0/minicluster-data/master-0/wal
uuid: "626ee287629d41f0a618efa008133984"
format_stamp: "Formatted at 2026-08-11 18:57:27 on dist-test-slave-h6n0"
server_key: "41ce5816a4869487411a8cd1e0923900"
server_key_iv: "cf26aa669b3dbe04907a6b4e21f43bfc"
server_key_version: "encryptionkey@0"
I20260811 18:57:27.405493 2952 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474615205622-30146-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474615205622-30146-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474615205622-30146-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:57:27.425908 2952 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:57:27.431384 2952 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:57:27.431757 2952 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:57:27.454651 2952 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.190:39197
I20260811 18:57:27.454700 3071 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.190:39197 every 8 connection(s)
I20260811 18:57:27.456187 2952 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
I20260811 18:57:27.460415 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 2952
I20260811 18:57:27.460718 3072 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:27.460744 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474615205622-30146-0/minicluster-data/master-0/wal/instance
I20260811 18:57:27.461665 30146 external_mini_cluster.cc:1428] Setting key 6be4723c8eacbead6b30a6fbcab8132a
I20260811 18:57:27.479333 3072 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 626ee287629d41f0a618efa008133984: Bootstrap starting.
Aug 11 18:57:27 dist-test-slave-h6n0 krb5kdc[2928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474646, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 18:57:27.483639 3072 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 626ee287629d41f0a618efa008133984: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:27.484846 3072 log.cc:824] T 00000000000000000000000000000000 P 626ee287629d41f0a618efa008133984: Log is configured to *not* fsync() on all Append() calls
I20260811 18:57:27.488204 3072 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 626ee287629d41f0a618efa008133984: No bootstrap required, opened a new log
I20260811 18:57:27.492477 3076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:27.464288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37090 (local address 127.29.112.190:39197)
0811 18:57:27.465464 (+ 1176us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:27.465480 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:27.465541 (+ 61us) server_negotiation.cc:407] Connection header received
0811 18:57:27.468333 (+ 2792us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:27.468364 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:27.468824 (+ 460us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:27.469212 (+ 388us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:27.471923 (+ 2711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:27.473449 (+ 1526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:27.475625 (+ 2176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:27.477053 (+ 1428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:27.477218 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:27.483258 (+ 6040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:27.483267 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:27.483283 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:27.483333 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:27.486940 (+ 3607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:27.487760 (+ 820us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:27.487767 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:27.487773 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:27.487950 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:27.488453 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:27.488459 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:27.488463 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:27.489575 (+ 1112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:27.489944 (+ 369us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:27.490346 (+ 402us) server_negotiation.cc:299] Negotiation successful
0811 18:57:27.490906 (+ 560us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":889,"thread_start_us":606,"threads_started":1}
I20260811 18:57:27.497507 30146 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:57:27.500553 3072 raft_consensus.cc:348] T 00000000000000000000000000000000 P 626ee287629d41f0a618efa008133984 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "626ee287629d41f0a618efa008133984" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 39197 } }
I20260811 18:57:27.501132 3072 raft_consensus.cc:374] T 00000000000000000000000000000000 P 626ee287629d41f0a618efa008133984 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:27.501264 3072 raft_consensus.cc:729] T 00000000000000000000000000000000 P 626ee287629d41f0a618efa008133984 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 626ee287629d41f0a618efa008133984, State: Initialized, Role: FOLLOWER
I20260811 18:57:27.502053 3072 consensus_queue.cc:260] T 00000000000000000000000000000000 P 626ee287629d41f0a618efa008133984 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "626ee287629d41f0a618efa008133984" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 39197 } }
I20260811 18:57:27.502396 3072 raft_consensus.cc:388] T 00000000000000000000000000000000 P 626ee287629d41f0a618efa008133984 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:57:27.502616 3072 raft_consensus.cc:482] T 00000000000000000000000000000000 P 626ee287629d41f0a618efa008133984 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:57:27.502841 3072 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 626ee287629d41f0a618efa008133984 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:27.507515 3072 raft_consensus.cc:504] T 00000000000000000000000000000000 P 626ee287629d41f0a618efa008133984 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "626ee287629d41f0a618efa008133984" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 39197 } }
I20260811 18:57:27.508288 3072 leader_election.cc:302] T 00000000000000000000000000000000 P 626ee287629d41f0a618efa008133984 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 626ee287629d41f0a618efa008133984; no voters:
I20260811 18:57:27.509169 3072 leader_election.cc:288] T 00000000000000000000000000000000 P 626ee287629d41f0a618efa008133984 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:57:27.509266 3083 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 626ee287629d41f0a618efa008133984 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:27.510640 3083 raft_consensus.cc:686] T 00000000000000000000000000000000 P 626ee287629d41f0a618efa008133984 [term 1 LEADER]: Becoming Leader. State: Replica: 626ee287629d41f0a618efa008133984, State: Running, Role: LEADER
I20260811 18:57:27.511461 3083 consensus_queue.cc:237] T 00000000000000000000000000000000 P 626ee287629d41f0a618efa008133984 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "626ee287629d41f0a618efa008133984" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 39197 } }
I20260811 18:57:27.512528 3072 sys_catalog.cc:564] T 00000000000000000000000000000000 P 626ee287629d41f0a618efa008133984 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:57:27.515883 3087 sys_catalog.cc:455] T 00000000000000000000000000000000 P 626ee287629d41f0a618efa008133984 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "626ee287629d41f0a618efa008133984" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "626ee287629d41f0a618efa008133984" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 39197 } } }
I20260811 18:57:27.516361 3087 sys_catalog.cc:458] T 00000000000000000000000000000000 P 626ee287629d41f0a618efa008133984 [sys.catalog]: This master's current role is: LEADER
I20260811 18:57:27.518456 3089 sys_catalog.cc:455] T 00000000000000000000000000000000 P 626ee287629d41f0a618efa008133984 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 626ee287629d41f0a618efa008133984. Latest consensus state: current_term: 1 leader_uuid: "626ee287629d41f0a618efa008133984" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "626ee287629d41f0a618efa008133984" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 39197 } } }
I20260811 18:57:27.518867 3089 sys_catalog.cc:458] T 00000000000000000000000000000000 P 626ee287629d41f0a618efa008133984 [sys.catalog]: This master's current role is: LEADER
I20260811 18:57:27.521757 3092 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:57:27.529803 3092 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:57:27.541385 3092 catalog_manager.cc:1347] Generated new cluster ID: 59aa269f5d194d3abdeed951783d1a31
I20260811 18:57:27.541550 3092 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:57:27.557413 3092 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:57:27.559522 3092 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:57:27.570109 3092 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 626ee287629d41f0a618efa008133984: Generated new TSK 0
I20260811 18:57:27.571302 3092 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:57:27.574754 3076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:27.564159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37104 (local address 127.29.112.190:39197)
0811 18:57:27.564332 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:27.564337 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:27.565359 (+ 1022us) server_negotiation.cc:407] Connection header received
0811 18:57:27.565518 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:27.565524 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:27.565621 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:27.565769 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:27.567153 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:27.568490 (+ 1337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:27.570384 (+ 1894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:27.571111 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:27.571227 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:27.572789 (+ 1562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:27.572797 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:27.572802 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:57:27.573629 (+ 827us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:27.573712 (+ 83us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:57:27.573750 (+ 38us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:27.573885 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:27.574236 (+ 351us) server_negotiation.cc:299] Negotiation successful
0811 18:57:27.574351 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 18:57:27.583668 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 2952
I20260811 18:57:27.584939 3063 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:57:27.679965 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 2952
2026-08-11T18:57:27Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (2331 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474615205622-30146-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:57:27 dist-test-slave-h6n0 krb5kdc[3128](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:57:27 dist-test-slave-h6n0 krb5kdc[3128](info): set up 2 sockets
Aug 11 18:57:27 dist-test-slave-h6n0 krb5kdc[3128](info): commencing operation
krb5kdc: starting...
W20260811 18:57:29.106666 30146 mini_kdc.cc:121] Time spent starting KDC: real 1.379s user 0.002s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474615205622-30146-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474615205622-30146-0/krb5kdc/test-user.keytab.
Aug 11 18:57:29 dist-test-slave-h6n0 krb5kdc[3128](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474649, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:57:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:57:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.112.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab.
Entry for principal kudu/127.29.112.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab.
I20260811 18:57:29.342237 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474615205622-30146-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474615205622-30146-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474615205622-30146-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.190:43525
--webserver_interface=127.29.112.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab
--principal=kudu/127.29.112.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:46089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.190:43525
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:57:29.680699 3152 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:57:29.680984 3152 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:57:29.681156 3152 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:57:29.688771 3152 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:57:29.688875 3152 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:57:29.689002 3152 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:57:29.689137 3152 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:57:29.689224 3152 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:57:29.699757 3152 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:46089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474615205622-30146-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474615205622-30146-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.112.190:43525
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab
--principal=kudu/127.29.112.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.190:43525
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.112.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474615205622-30146-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:57:29.701133 3152 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:57:29.702598 3152 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:57:29.717101 3158 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:29.717036 3162 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:29.717283 3160 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:57:29.719719 3152 server_base.cc:1034] running on GCE node
I20260811 18:57:29.720665 3152 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:57:29.722214 3152 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:57:29.723433 3152 hybrid_clock.cc:648] HybridClock initialized: now 1786474649723388 us; error 46 us; skew 500 ppm
I20260811 18:57:29.723858 3152 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:57:29 dist-test-slave-h6n0 krb5kdc[3128](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474649, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:57:29.731655 3152 init.cc:377] Logged in from keytab as kudu/127.29.112.190@KRBTEST.COM (short username kudu)
I20260811 18:57:29.733323 3152 webserver.cc:460] Webserver started at http://127.29.112.190:39433/ using document root <none> and password file <none>
I20260811 18:57:29.734054 3152 fs_manager.cc:359] Metadata directory not provided
I20260811 18:57:29.734211 3152 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:57:29.734599 3152 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:57:29.738030 3152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474615205622-30146-0/minicluster-data/master-0/data/instance:
uuid: "cb902e0f77094facb4337d2386679336"
format_stamp: "Formatted at 2026-08-11 18:57:29 on dist-test-slave-h6n0"
server_key: "cc0cdf623b03391e14bdaa70c2d494c4"
server_key_iv: "f249db11ada3fd3d5e1b64cdeacb5803"
server_key_version: "encryptionkey@0"
I20260811 18:57:29.739190 3152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474615205622-30146-0/minicluster-data/master-0/wal/instance:
uuid: "cb902e0f77094facb4337d2386679336"
format_stamp: "Formatted at 2026-08-11 18:57:29 on dist-test-slave-h6n0"
server_key: "cc0cdf623b03391e14bdaa70c2d494c4"
server_key_iv: "f249db11ada3fd3d5e1b64cdeacb5803"
server_key_version: "encryptionkey@0"
I20260811 18:57:29.746662 3152 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.002s sys 0.004s
I20260811 18:57:29.750778 3171 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:29.752213 3152 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:57:29.752367 3152 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474615205622-30146-0/minicluster-data/master-0/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474615205622-30146-0/minicluster-data/master-0/wal
uuid: "cb902e0f77094facb4337d2386679336"
format_stamp: "Formatted at 2026-08-11 18:57:29 on dist-test-slave-h6n0"
server_key: "cc0cdf623b03391e14bdaa70c2d494c4"
server_key_iv: "f249db11ada3fd3d5e1b64cdeacb5803"
server_key_version: "encryptionkey@0"
I20260811 18:57:29.752610 3152 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474615205622-30146-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474615205622-30146-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474615205622-30146-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:57:29.790628 3152 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:57:29.794879 3152 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:57:29.795182 3152 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:57:29.820083 3152 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.190:43525
I20260811 18:57:29.820109 3256 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.190:43525 every 8 connection(s)
I20260811 18:57:29.822078 3152 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
I20260811 18:57:29.825007 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 3152
I20260811 18:57:29.825253 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474615205622-30146-0/minicluster-data/master-0/wal/instance
I20260811 18:57:29.825965 30146 external_mini_cluster.cc:1428] Setting key e626f548112913343e97805ae8febeee
I20260811 18:57:29.826110 3257 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:29.840953 3257 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb902e0f77094facb4337d2386679336: Bootstrap starting.
Aug 11 18:57:29 dist-test-slave-h6n0 krb5kdc[3128](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474649, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 18:57:29.844928 3257 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cb902e0f77094facb4337d2386679336: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:29.846479 3257 log.cc:824] T 00000000000000000000000000000000 P cb902e0f77094facb4337d2386679336: Log is configured to *not* fsync() on all Append() calls
I20260811 18:57:29.850391 3257 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb902e0f77094facb4337d2386679336: No bootstrap required, opened a new log
I20260811 18:57:29.852411 3261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:29.828193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43664 (local address 127.29.112.190:43525)
0811 18:57:29.829120 (+ 927us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:29.829135 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:29.829188 (+ 53us) server_negotiation.cc:407] Connection header received
0811 18:57:29.831340 (+ 2152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:29.831367 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:29.831866 (+ 499us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:29.832298 (+ 432us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:29.833614 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:29.835926 (+ 2312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:29.837240 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:29.838325 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:29.838508 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:29.842855 (+ 4347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:29.842869 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:29.842889 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:29.842948 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:29.847198 (+ 4250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:29.848255 (+ 1057us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:29.848264 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:29.848272 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:29.848483 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:29.848954 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:29.848959 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:29.848962 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:29.849678 (+ 716us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:29.850104 (+ 426us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:29.850601 (+ 497us) server_negotiation.cc:299] Negotiation successful
0811 18:57:29.851070 (+ 469us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":659,"thread_start_us":367,"threads_started":1}
I20260811 18:57:29.860046 3257 raft_consensus.cc:348] T 00000000000000000000000000000000 P cb902e0f77094facb4337d2386679336 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb902e0f77094facb4337d2386679336" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 43525 } }
I20260811 18:57:29.860406 3257 raft_consensus.cc:374] T 00000000000000000000000000000000 P cb902e0f77094facb4337d2386679336 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:29.860522 3257 raft_consensus.cc:729] T 00000000000000000000000000000000 P cb902e0f77094facb4337d2386679336 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb902e0f77094facb4337d2386679336, State: Initialized, Role: FOLLOWER
I20260811 18:57:29.861111 3257 consensus_queue.cc:260] T 00000000000000000000000000000000 P cb902e0f77094facb4337d2386679336 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb902e0f77094facb4337d2386679336" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 43525 } }
I20260811 18:57:29.861318 3257 raft_consensus.cc:388] T 00000000000000000000000000000000 P cb902e0f77094facb4337d2386679336 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:57:29.861497 3257 raft_consensus.cc:482] T 00000000000000000000000000000000 P cb902e0f77094facb4337d2386679336 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:57:29.861676 3257 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cb902e0f77094facb4337d2386679336 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:29.864043 3257 raft_consensus.cc:504] T 00000000000000000000000000000000 P cb902e0f77094facb4337d2386679336 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb902e0f77094facb4337d2386679336" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 43525 } }
I20260811 18:57:29.864583 3257 leader_election.cc:302] T 00000000000000000000000000000000 P cb902e0f77094facb4337d2386679336 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cb902e0f77094facb4337d2386679336; no voters:
I20260811 18:57:29.865522 3257 leader_election.cc:288] T 00000000000000000000000000000000 P cb902e0f77094facb4337d2386679336 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:57:29.865871 3267 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cb902e0f77094facb4337d2386679336 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:29.866945 3267 raft_consensus.cc:686] T 00000000000000000000000000000000 P cb902e0f77094facb4337d2386679336 [term 1 LEADER]: Becoming Leader. State: Replica: cb902e0f77094facb4337d2386679336, State: Running, Role: LEADER
I20260811 18:57:29.867664 3267 consensus_queue.cc:237] T 00000000000000000000000000000000 P cb902e0f77094facb4337d2386679336 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb902e0f77094facb4337d2386679336" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 43525 } }
I20260811 18:57:29.867978 3257 sys_catalog.cc:564] T 00000000000000000000000000000000 P cb902e0f77094facb4337d2386679336 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:57:29.871709 3268 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb902e0f77094facb4337d2386679336 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cb902e0f77094facb4337d2386679336" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb902e0f77094facb4337d2386679336" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 43525 } } }
I20260811 18:57:29.871728 3269 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb902e0f77094facb4337d2386679336 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cb902e0f77094facb4337d2386679336. Latest consensus state: current_term: 1 leader_uuid: "cb902e0f77094facb4337d2386679336" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb902e0f77094facb4337d2386679336" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 43525 } } }
I20260811 18:57:29.872287 3269 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb902e0f77094facb4337d2386679336 [sys.catalog]: This master's current role is: LEADER
I20260811 18:57:29.872287 3268 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb902e0f77094facb4337d2386679336 [sys.catalog]: This master's current role is: LEADER
I20260811 18:57:29.874675 3282 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:57:29.879160 3282 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:57:29.889338 3282 catalog_manager.cc:1347] Generated new cluster ID: a3b3c155fac943a7adc9e99931a6e603
I20260811 18:57:29.889519 3282 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:57:29.919658 3282 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:57:29.921330 3282 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:57:29.927933 3282 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cb902e0f77094facb4337d2386679336: Generated new TSK 0
I20260811 18:57:29.928797 3282 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.112.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.129.keytab.
Entry for principal kudu/127.29.112.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.129.keytab.
I20260811 18:57:29.994635 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.129:0
--local_ip_for_outbound_sockets=127.29.112.129
--webserver_interface=127.29.112.129
--webserver_port=0
--tserver_master_addrs=127.29.112.190:43525
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.129.keytab
--principal=kudu/127.29.112.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:46089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:57:30.343550 3302 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:57:30.343828 3302 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:57:30.343936 3302 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:57:30.351243 3302 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:57:30.351419 3302 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:57:30.351658 3302 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.129
I20260811 18:57:30.361713 3302 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:46089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.129.keytab
--principal=kudu/127.29.112.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.112.129
--webserver_port=0
--tserver_master_addrs=127.29.112.190:43525
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.129
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:57:30.362993 3302 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:57:30.364250 3302 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:57:30.378255 3310 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:30.378156 3315 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:30.378638 3312 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:57:30.379643 3302 server_base.cc:1034] running on GCE node
I20260811 18:57:30.380391 3302 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:57:30.381417 3302 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:57:30.382660 3302 hybrid_clock.cc:648] HybridClock initialized: now 1786474650382620 us; error 38 us; skew 500 ppm
I20260811 18:57:30.383008 3302 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:57:30 dist-test-slave-h6n0 krb5kdc[3128](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474650, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:57:30.386512 3302 init.cc:377] Logged in from keytab as kudu/127.29.112.129@KRBTEST.COM (short username kudu)
I20260811 18:57:30.388033 3302 webserver.cc:460] Webserver started at http://127.29.112.129:43141/ using document root <none> and password file <none>
I20260811 18:57:30.388746 3302 fs_manager.cc:359] Metadata directory not provided
I20260811 18:57:30.388916 3302 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:57:30.389254 3302 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:57:30.392380 3302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-0/data/instance:
uuid: "41ee0252999f4725b62239e34e29e779"
format_stamp: "Formatted at 2026-08-11 18:57:30 on dist-test-slave-h6n0"
server_key: "38f4e73ca71c08ba91c81cc43bdab821"
server_key_iv: "53be38ad97a64e7823e0b4bc0c39de4f"
server_key_version: "encryptionkey@0"
I20260811 18:57:30.393253 3302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-0/wal/instance:
uuid: "41ee0252999f4725b62239e34e29e779"
format_stamp: "Formatted at 2026-08-11 18:57:30 on dist-test-slave-h6n0"
server_key: "38f4e73ca71c08ba91c81cc43bdab821"
server_key_iv: "53be38ad97a64e7823e0b4bc0c39de4f"
server_key_version: "encryptionkey@0"
I20260811 18:57:30.398989 3302 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 18:57:30.402573 3327 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:30.403930 3302 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 18:57:30.404122 3302 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-0/wal
uuid: "41ee0252999f4725b62239e34e29e779"
format_stamp: "Formatted at 2026-08-11 18:57:30 on dist-test-slave-h6n0"
server_key: "38f4e73ca71c08ba91c81cc43bdab821"
server_key_iv: "53be38ad97a64e7823e0b4bc0c39de4f"
server_key_version: "encryptionkey@0"
I20260811 18:57:30.404328 3302 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:57:30.415951 3302 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:57:30.420313 3302 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:57:30.420639 3302 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:57:30.421620 3302 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:57:30.423192 3302 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:57:30.423301 3302 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:30.423456 3302 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:57:30.423530 3302 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:30.464198 3302 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.129:34679
I20260811 18:57:30.464236 3499 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.129:34679 every 8 connection(s)
I20260811 18:57:30.465562 3302 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-0/data/info.pb
I20260811 18:57:30.469601 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 3302
I20260811 18:57:30.469934 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-0/wal/instance
I20260811 18:57:30.470944 30146 external_mini_cluster.cc:1428] Setting key 12decd168d362290bbe236ee11f0920b
Aug 11 18:57:30 dist-test-slave-h6n0 krb5kdc[3128](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474650, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.129@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 18:57:30.491840 3505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:30.468879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.129:40969 (local address 127.29.112.190:43525)
0811 18:57:30.469606 (+ 727us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:30.469612 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:30.470887 (+ 1275us) server_negotiation.cc:407] Connection header received
0811 18:57:30.474040 (+ 3153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:30.474046 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:30.474151 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:30.474281 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:30.476317 (+ 2036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:30.477248 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:30.479444 (+ 2196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:30.479932 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:30.480082 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:30.484619 (+ 4537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:30.484624 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:30.484628 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:30.484667 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:30.487586 (+ 2919us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:30.488425 (+ 839us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:30.488430 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:30.488432 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:30.488558 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:30.489030 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:30.489048 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:30.489051 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:30.489330 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:30.489493 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:30.490840 (+ 1347us) server_negotiation.cc:299] Negotiation successful
0811 18:57:30.491337 (+ 497us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":586,"thread_start_us":337,"threads_started":1}
I20260811 18:57:30.493119 3507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:30.469429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:43525 (local address 127.29.112.129:40969)
0811 18:57:30.470656 (+ 1227us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:30.470736 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:30.473552 (+ 2816us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:30.474599 (+ 1047us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:30.474613 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:30.475535 (+ 922us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:57:30.476132 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:30.476150 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:30.477525 (+ 1375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:30.477531 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:30.479239 (+ 1708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:30.479263 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:30.480256 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:30.480265 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:30.480551 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:30.481482 (+ 931us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:57:30.481511 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:57:30.484330 (+ 2819us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:57:30.487750 (+ 3420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:30.487761 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:30.487778 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:30.488275 (+ 497us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:30.488680 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:30.488685 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:30.488689 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:30.488891 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:30.489469 (+ 578us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:57:30.489477 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:57:30.490135 (+ 658us) client_negotiation.cc:769] Sending connection context
0811 18:57:30.490704 (+ 569us) client_negotiation.cc:240] Negotiation successful
0811 18:57:30.491591 (+ 887us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":810,"thread_start_us":496,"threads_started":1}
WARNING: no policy specified for kudu/127.29.112.130@KRBTEST.COM; defaulting to no policy
I20260811 18:57:30.495525 3501 heartbeater.cc:347] Connected to a master server at 127.29.112.190:43525
I20260811 18:57:30.496001 3501 heartbeater.cc:464] Registering TS with master...
I20260811 18:57:30.497062 3501 heartbeater.cc:511] Master 127.29.112.190:43525 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.130@KRBTEST.COM" created.
I20260811 18:57:30.499970 3200 ts_manager.cc:194] Registered new tserver with Master: 41ee0252999f4725b62239e34e29e779 (127.29.112.129:34679)
I20260811 18:57:30.501962 3200 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.129@KRBTEST.COM'} at 127.29.112.129:40969
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.130.keytab.
Entry for principal kudu/127.29.112.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.130.keytab.
I20260811 18:57:30.531463 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.130:0
--local_ip_for_outbound_sockets=127.29.112.130
--webserver_interface=127.29.112.130
--webserver_port=0
--tserver_master_addrs=127.29.112.190:43525
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.130.keytab
--principal=kudu/127.29.112.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:46089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:57:30.911319 3511 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:57:30.911652 3511 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:57:30.911820 3511 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:57:30.919883 3511 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:57:30.920025 3511 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:57:30.920253 3511 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.130
I20260811 18:57:30.931983 3511 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:46089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.130.keytab
--principal=kudu/127.29.112.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.112.130
--webserver_port=0
--tserver_master_addrs=127.29.112.190:43525
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.130
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:57:30.933336 3511 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:57:30.934937 3511 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:57:30.951212 3519 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:30.951257 3525 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:30.951171 3523 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:57:30.952116 3511 server_base.cc:1034] running on GCE node
I20260811 18:57:30.952834 3511 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:57:30.953827 3511 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:57:30.955103 3511 hybrid_clock.cc:648] HybridClock initialized: now 1786474650955063 us; error 62 us; skew 500 ppm
I20260811 18:57:30.955454 3511 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:57:30 dist-test-slave-h6n0 krb5kdc[3128](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474650, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:57:30.959117 3511 init.cc:377] Logged in from keytab as kudu/127.29.112.130@KRBTEST.COM (short username kudu)
I20260811 18:57:30.960554 3511 webserver.cc:460] Webserver started at http://127.29.112.130:34191/ using document root <none> and password file <none>
I20260811 18:57:30.961202 3511 fs_manager.cc:359] Metadata directory not provided
I20260811 18:57:30.961308 3511 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:57:30.961570 3511 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:57:30.964419 3511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-1/data/instance:
uuid: "5c08d364c45b47578057b1d7144e317d"
format_stamp: "Formatted at 2026-08-11 18:57:30 on dist-test-slave-h6n0"
server_key: "4fa7bd873ec253c8c1f49eb8b545295e"
server_key_iv: "2df00c48cfac420d4847494e84314dc1"
server_key_version: "encryptionkey@0"
I20260811 18:57:30.965145 3511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-1/wal/instance:
uuid: "5c08d364c45b47578057b1d7144e317d"
format_stamp: "Formatted at 2026-08-11 18:57:30 on dist-test-slave-h6n0"
server_key: "4fa7bd873ec253c8c1f49eb8b545295e"
server_key_iv: "2df00c48cfac420d4847494e84314dc1"
server_key_version: "encryptionkey@0"
I20260811 18:57:30.970315 3511 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260811 18:57:30.973740 3534 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:30.975168 3511 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:57:30.975303 3511 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-1/wal
uuid: "5c08d364c45b47578057b1d7144e317d"
format_stamp: "Formatted at 2026-08-11 18:57:30 on dist-test-slave-h6n0"
server_key: "4fa7bd873ec253c8c1f49eb8b545295e"
server_key_iv: "2df00c48cfac420d4847494e84314dc1"
server_key_version: "encryptionkey@0"
I20260811 18:57:30.975467 3511 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:57:30.992314 3511 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:57:30.997267 3511 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:57:30.997553 3511 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:57:30.998772 3511 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:57:31.000490 3511 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:57:31.000581 3511 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:31.000679 3511 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:57:31.000741 3511 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:31.039832 3511 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.130:34625
I20260811 18:57:31.039871 3725 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.130:34625 every 8 connection(s)
I20260811 18:57:31.041152 3511 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-1/data/info.pb
I20260811 18:57:31.048743 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 3511
I20260811 18:57:31.048982 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-1/wal/instance
I20260811 18:57:31.049466 30146 external_mini_cluster.cc:1428] Setting key 658d97ad14e879e2ebdeb4929f6f0374
Aug 11 18:57:31 dist-test-slave-h6n0 krb5kdc[3128](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474650, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.130@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 18:57:31.065178 3729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:31.044430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.130:40837 (local address 127.29.112.190:43525)
0811 18:57:31.044921 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:31.044926 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:31.046051 (+ 1125us) server_negotiation.cc:407] Connection header received
0811 18:57:31.048280 (+ 2229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:31.048286 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:31.048406 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:31.048544 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:31.050333 (+ 1789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:31.051194 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:31.052953 (+ 1759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:31.053416 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:31.053575 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:31.057695 (+ 4120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:31.057700 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:31.057703 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:31.057736 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:31.061618 (+ 3882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:31.062495 (+ 877us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:31.062503 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:31.062507 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:31.062704 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:31.063203 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:31.063209 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:31.063213 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:31.063527 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:31.063697 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:31.064478 (+ 781us) server_negotiation.cc:299] Negotiation successful
0811 18:57:31.064736 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":258,"threads_started":1}
I20260811 18:57:31.066283 3731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:31.045003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:43525 (local address 127.29.112.130:40837)
0811 18:57:31.045853 (+ 850us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:31.045909 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:31.047941 (+ 2032us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:31.048803 (+ 862us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:31.048835 (+ 32us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:31.049466 (+ 631us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:57:31.050110 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:31.050128 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:31.051470 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:31.051475 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:31.052769 (+ 1294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:31.052782 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:31.053673 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:31.053680 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:31.053944 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:31.054765 (+ 821us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:57:31.054792 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:57:31.057412 (+ 2620us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:57:31.061825 (+ 4413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:31.061836 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:31.061857 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:31.062341 (+ 484us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:31.062835 (+ 494us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:31.062840 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:31.062845 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:31.063061 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:31.063687 (+ 626us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:57:31.063695 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:57:31.064071 (+ 376us) client_negotiation.cc:769] Sending connection context
0811 18:57:31.064390 (+ 319us) client_negotiation.cc:240] Negotiation successful
0811 18:57:31.064978 (+ 588us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":584,"thread_start_us":354,"threads_started":1}
I20260811 18:57:31.068411 3726 heartbeater.cc:347] Connected to a master server at 127.29.112.190:43525
I20260811 18:57:31.068769 3726 heartbeater.cc:464] Registering TS with master...
I20260811 18:57:31.069579 3726 heartbeater.cc:511] Master 127.29.112.190:43525 requested a full tablet report, sending...
I20260811 18:57:31.071501 3200 ts_manager.cc:194] Registered new tserver with Master: 5c08d364c45b47578057b1d7144e317d (127.29.112.130:34625)
I20260811 18:57:31.072878 3200 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.130@KRBTEST.COM'} at 127.29.112.130:40837
WARNING: no policy specified for kudu/127.29.112.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.131.keytab.
Entry for principal kudu/127.29.112.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.131.keytab.
I20260811 18:57:31.106751 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.131:0
--local_ip_for_outbound_sockets=127.29.112.131
--webserver_interface=127.29.112.131
--webserver_port=0
--tserver_master_addrs=127.29.112.190:43525
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.131.keytab
--principal=kudu/127.29.112.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:46089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:57:31.446002 3738 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:57:31.446271 3738 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:57:31.446367 3738 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:57:31.453476 3738 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:57:31.453588 3738 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:57:31.453701 3738 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.131
I20260811 18:57:31.463958 3738 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:46089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.131.keytab
--principal=kudu/127.29.112.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.112.131
--webserver_port=0
--tserver_master_addrs=127.29.112.190:43525
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.131
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:57:31.466127 3738 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:57:31.468427 3738 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:57:31.485526 3744 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:31.485562 3751 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:31.485637 3749 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:57:31.486236 3738 server_base.cc:1034] running on GCE node
I20260811 18:57:31.486917 3738 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:57:31.487816 3738 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:57:31.489014 3738 hybrid_clock.cc:648] HybridClock initialized: now 1786474651488949 us; error 69 us; skew 500 ppm
I20260811 18:57:31.489321 3738 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:57:31 dist-test-slave-h6n0 krb5kdc[3128](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474651, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:57:31.492636 3738 init.cc:377] Logged in from keytab as kudu/127.29.112.131@KRBTEST.COM (short username kudu)
I20260811 18:57:31.494174 3738 webserver.cc:460] Webserver started at http://127.29.112.131:35447/ using document root <none> and password file <none>
I20260811 18:57:31.494942 3738 fs_manager.cc:359] Metadata directory not provided
I20260811 18:57:31.495103 3738 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:57:31.495426 3738 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:57:31.498317 3738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-2/data/instance:
uuid: "3b32eb2875b6483ab315e65b07ffc3b9"
format_stamp: "Formatted at 2026-08-11 18:57:31 on dist-test-slave-h6n0"
server_key: "b453cfb9bc53b590624645a0e74f4446"
server_key_iv: "32073c9d507464b3431c08fcc1527a98"
server_key_version: "encryptionkey@0"
I20260811 18:57:31.499531 3738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-2/wal/instance:
uuid: "3b32eb2875b6483ab315e65b07ffc3b9"
format_stamp: "Formatted at 2026-08-11 18:57:31 on dist-test-slave-h6n0"
server_key: "b453cfb9bc53b590624645a0e74f4446"
server_key_iv: "32073c9d507464b3431c08fcc1527a98"
server_key_version: "encryptionkey@0"
I20260811 18:57:31.505328 3738 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260811 18:57:31.506827 3501 heartbeater.cc:503] Master 127.29.112.190:43525 was elected leader, sending a full tablet report...
I20260811 18:57:31.509598 3768 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:31.511282 3738 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 18:57:31.511489 3738 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-2/wal
uuid: "3b32eb2875b6483ab315e65b07ffc3b9"
format_stamp: "Formatted at 2026-08-11 18:57:31 on dist-test-slave-h6n0"
server_key: "b453cfb9bc53b590624645a0e74f4446"
server_key_iv: "32073c9d507464b3431c08fcc1527a98"
server_key_version: "encryptionkey@0"
I20260811 18:57:31.511713 3738 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:57:31.534078 3738 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:57:31.539724 3738 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:57:31.540117 3738 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:57:31.541227 3738 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:57:31.542788 3738 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:57:31.542892 3738 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:31.543025 3738 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:57:31.543103 3738 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:31.583282 3738 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.131:38029
I20260811 18:57:31.583326 3954 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.131:38029 every 8 connection(s)
I20260811 18:57:31.584666 3738 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-2/data/info.pb
I20260811 18:57:31.592697 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 3738
I20260811 18:57:31.592906 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474615205622-30146-0/minicluster-data/ts-2/wal/instance
I20260811 18:57:31.593408 30146 external_mini_cluster.cc:1428] Setting key 9e79e59396799fba486c6f8acd656e6c
Aug 11 18:57:31 dist-test-slave-h6n0 krb5kdc[3128](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474651, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.131@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 18:57:31.606276 3958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:31.587384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.131:46745 (local address 127.29.112.190:43525)
0811 18:57:31.587862 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:31.587870 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:31.588844 (+ 974us) server_negotiation.cc:407] Connection header received
0811 18:57:31.590669 (+ 1825us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:31.590674 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:31.590742 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:31.590849 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:31.592260 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:31.593233 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:31.595912 (+ 2679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:31.596413 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:31.596547 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:31.600392 (+ 3845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:31.600399 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:31.600404 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:31.600456 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:31.603703 (+ 3247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:31.604387 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:31.604391 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:31.604393 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:31.604523 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:31.604897 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:31.604901 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:31.604903 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:31.605166 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:31.605285 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:31.605843 (+ 558us) server_negotiation.cc:299] Negotiation successful
0811 18:57:31.605998 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":231,"threads_started":1}
I20260811 18:57:31.607218 3959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:31.587949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:43525 (local address 127.29.112.131:46745)
0811 18:57:31.588685 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:31.588723 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:31.590337 (+ 1614us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:31.591053 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:31.591061 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:31.591551 (+ 490us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:57:31.592081 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:31.592097 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:31.593387 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:31.593395 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:31.595723 (+ 2328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:31.595740 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:31.596653 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:31.596657 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:31.596872 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:31.597807 (+ 935us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:57:31.597836 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:57:31.600107 (+ 2271us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:57:31.603826 (+ 3719us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:31.603833 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:31.603843 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:31.604250 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:31.604621 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:31.604624 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:31.604627 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:31.604792 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:31.605271 (+ 479us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:57:31.605276 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:57:31.605550 (+ 274us) client_negotiation.cc:769] Sending connection context
0811 18:57:31.605765 (+ 215us) client_negotiation.cc:240] Negotiation successful
0811 18:57:31.606118 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":287,"threads_started":1}
I20260811 18:57:31.608707 3956 heartbeater.cc:347] Connected to a master server at 127.29.112.190:43525
I20260811 18:57:31.609041 3956 heartbeater.cc:464] Registering TS with master...
I20260811 18:57:31.609733 3956 heartbeater.cc:511] Master 127.29.112.190:43525 requested a full tablet report, sending...
I20260811 18:57:31.611097 3200 ts_manager.cc:194] Registered new tserver with Master: 3b32eb2875b6483ab315e65b07ffc3b9 (127.29.112.131:38029)
I20260811 18:57:31.611887 3200 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.131@KRBTEST.COM'} at 127.29.112.131:46745
I20260811 18:57:31.622208 30146 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 18:57:31 dist-test-slave-h6n0 krb5kdc[3128](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474651, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 11 18:57:31 dist-test-slave-h6n0 krb5kdc[3128](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474651, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 18:57:31.682461 3958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:31.666124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43666 (local address 127.29.112.190:43525)
0811 18:57:31.666410 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:31.666419 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:31.666876 (+ 457us) server_negotiation.cc:407] Connection header received
0811 18:57:31.667098 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:31.667111 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:31.667181 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:31.667298 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:31.668889 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:31.669685 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:31.671041 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:31.671561 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:31.671738 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:31.676866 (+ 5128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:31.676871 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:31.676874 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:31.676910 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:31.679665 (+ 2755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:31.680318 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:31.680322 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:31.680325 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:31.680442 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:31.680832 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:31.680837 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:31.680839 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:31.681217 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:31.681422 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:31.681843 (+ 421us) server_negotiation.cc:299] Negotiation successful
0811 18:57:31.682018 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112}
W20260811 18:57:31.683091 3200 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:43666
I20260811 18:57:31.685122 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 3302
Aug 11 18:57:31 dist-test-slave-h6n0 krb5kdc[3128](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474651, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.29.112.129@KRBTEST.COM
I20260811 18:57:31.704151 3974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:31.686477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46496 (local address 127.29.112.129:34679)
0811 18:57:31.687443 (+ 966us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:31.687452 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:31.687484 (+ 32us) server_negotiation.cc:407] Connection header received
0811 18:57:31.687571 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:31.687581 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:31.687852 (+ 271us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:31.688017 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:31.689320 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:31.690799 (+ 1479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:31.692302 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:31.693087 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:31.693230 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:31.697532 (+ 4302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:31.697546 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:31.697560 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:31.697601 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:31.700663 (+ 3062us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:31.701433 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:31.701445 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:31.701447 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:31.701707 (+ 260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:31.702231 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:31.702236 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:31.702239 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:31.702762 (+ 523us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:31.703042 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:31.703502 (+ 460us) server_negotiation.cc:299] Negotiation successful
0811 18:57:31.703809 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":748,"thread_start_us":495,"threads_started":1}
W20260811 18:57:31.705551 3492 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:46496
W20260811 18:57:31.706861 30146 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 3302: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 18:57:31.707067 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 3302
I20260811 18:57:31.739437 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 3511
Aug 11 18:57:31 dist-test-slave-h6n0 krb5kdc[3128](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474651, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.29.112.130@KRBTEST.COM
I20260811 18:57:31.758479 3976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:31.740588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49668 (local address 127.29.112.130:34625)
0811 18:57:31.741461 (+ 873us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:31.741472 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:31.741513 (+ 41us) server_negotiation.cc:407] Connection header received
0811 18:57:31.741624 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:31.741633 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:31.741937 (+ 304us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:31.742151 (+ 214us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:31.743639 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:31.744705 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:31.746239 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:31.747032 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:31.747147 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:31.751949 (+ 4802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:31.751961 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:31.751976 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:31.752012 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:31.755157 (+ 3145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:31.755895 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:31.755903 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:31.755905 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:31.756038 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:31.756510 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:31.756514 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:31.756516 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:31.756890 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:31.757143 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:31.757883 (+ 740us) server_negotiation.cc:299] Negotiation successful
0811 18:57:31.758106 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":666,"thread_start_us":430,"threads_started":1}
W20260811 18:57:31.759825 3717 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:49668
W20260811 18:57:31.760938 30146 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 3511: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 18:57:31.761113 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 3511
I20260811 18:57:31.812520 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 3738
Aug 11 18:57:31 dist-test-slave-h6n0 krb5kdc[3128](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474651, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.29.112.131@KRBTEST.COM
I20260811 18:57:31.831940 3977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:31.813904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40890 (local address 127.29.112.131:38029)
0811 18:57:31.814761 (+ 857us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:31.814768 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:31.814832 (+ 64us) server_negotiation.cc:407] Connection header received
0811 18:57:31.814916 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:31.814923 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:31.815159 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:31.815299 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:31.816840 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:31.817933 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:31.819421 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:31.819986 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:31.820145 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:31.825240 (+ 5095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:31.825258 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:31.825278 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:31.825327 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:31.828823 (+ 3496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:31.829654 (+ 831us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:31.829662 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:31.829664 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:31.829794 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:31.830173 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:31.830179 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:31.830182 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:31.830611 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:31.830935 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:31.831273 (+ 338us) server_negotiation.cc:299] Negotiation successful
0811 18:57:31.831579 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":552,"thread_start_us":348,"threads_started":1}
W20260811 18:57:31.833738 3948 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:40890
W20260811 18:57:31.835103 30146 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 3738: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 18:57:31.835280 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 3738
I20260811 18:57:31.860399 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 3152
I20260811 18:57:31.861968 3252 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:57:31.971635 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 3152
2026-08-11T18:57:32Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (4298 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474615205622-30146-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:57:32 dist-test-slave-h6n0 krb5kdc[3983](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:57:32 dist-test-slave-h6n0 krb5kdc[3983](info): set up 2 sockets
Aug 11 18:57:32 dist-test-slave-h6n0 krb5kdc[3983](info): commencing operation
krb5kdc: starting...
W20260811 18:57:33.337149 30146 mini_kdc.cc:121] Time spent starting KDC: real 1.311s user 0.000s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474615205622-30146-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474615205622-30146-0/krb5kdc/test-user.keytab.
Aug 11 18:57:33 dist-test-slave-h6n0 krb5kdc[3983](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474653, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:57:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:57:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.112.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab.
Entry for principal kudu/127.29.112.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab.
I20260811 18:57:33.586885 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474615205622-30146-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474615205622-30146-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474615205622-30146-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.190:40737
--webserver_interface=127.29.112.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab
--principal=kudu/127.29.112.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:41957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.190:40737
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:57:33.973558 4003 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:57:33.973899 4003 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:57:33.974100 4003 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:57:33.982350 4003 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:57:33.982496 4003 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:57:33.982613 4003 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:57:33.982792 4003 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:57:33.982853 4003 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:57:33.993212 4003 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:41957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474615205622-30146-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474615205622-30146-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.112.190:40737
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab
--principal=kudu/127.29.112.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.190:40737
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.112.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474615205622-30146-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:57:33.994685 4003 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:57:33.996094 4003 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:57:34.012182 4010 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:34.012216 4015 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:34.012943 4013 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:57:34.014258 4003 server_base.cc:1034] running on GCE node
I20260811 18:57:34.015908 4003 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:57:34.017846 4003 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:57:34.019098 4003 hybrid_clock.cc:648] HybridClock initialized: now 1786474654019032 us; error 61 us; skew 500 ppm
I20260811 18:57:34.019521 4003 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:57:34 dist-test-slave-h6n0 krb5kdc[3983](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474654, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:57:34.022981 4003 init.cc:377] Logged in from keytab as kudu/127.29.112.190@KRBTEST.COM (short username kudu)
I20260811 18:57:34.024631 4003 webserver.cc:460] Webserver started at http://127.29.112.190:46287/ using document root <none> and password file <none>
I20260811 18:57:34.025321 4003 fs_manager.cc:359] Metadata directory not provided
I20260811 18:57:34.025425 4003 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:57:34.025704 4003 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:57:34.028815 4003 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474615205622-30146-0/minicluster-data/master-0/data/instance:
uuid: "5eab9167a5d8474698754966a3137b3e"
format_stamp: "Formatted at 2026-08-11 18:57:34 on dist-test-slave-h6n0"
server_key: "d5fd9a8b80ebd236f70a877f462f589e"
server_key_iv: "865f9d23900ccce9687850bcd650fa39"
server_key_version: "encryptionkey@0"
I20260811 18:57:34.029579 4003 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474615205622-30146-0/minicluster-data/master-0/wal/instance:
uuid: "5eab9167a5d8474698754966a3137b3e"
format_stamp: "Formatted at 2026-08-11 18:57:34 on dist-test-slave-h6n0"
server_key: "d5fd9a8b80ebd236f70a877f462f589e"
server_key_iv: "865f9d23900ccce9687850bcd650fa39"
server_key_version: "encryptionkey@0"
I20260811 18:57:34.035444 4003 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 18:57:34.040110 4029 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:34.041595 4003 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 18:57:34.041800 4003 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474615205622-30146-0/minicluster-data/master-0/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474615205622-30146-0/minicluster-data/master-0/wal
uuid: "5eab9167a5d8474698754966a3137b3e"
format_stamp: "Formatted at 2026-08-11 18:57:34 on dist-test-slave-h6n0"
server_key: "d5fd9a8b80ebd236f70a877f462f589e"
server_key_iv: "865f9d23900ccce9687850bcd650fa39"
server_key_version: "encryptionkey@0"
I20260811 18:57:34.042028 4003 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474615205622-30146-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474615205622-30146-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474615205622-30146-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:57:34.077133 4003 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:57:34.081633 4003 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:57:34.081959 4003 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:57:34.103070 4003 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.190:40737
I20260811 18:57:34.103087 4118 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.190:40737 every 8 connection(s)
I20260811 18:57:34.104518 4003 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
I20260811 18:57:34.105260 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 4003
I20260811 18:57:34.105522 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474615205622-30146-0/minicluster-data/master-0/wal/instance
I20260811 18:57:34.106419 30146 external_mini_cluster.cc:1428] Setting key ffd7b0a1aac1f81cdd20ad556c0572b4
I20260811 18:57:34.109572 4120 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 18:57:34 dist-test-slave-h6n0 krb5kdc[3983](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474653, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 18:57:34.120217 4120 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5eab9167a5d8474698754966a3137b3e: Bootstrap starting.
I20260811 18:57:34.123878 4120 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5eab9167a5d8474698754966a3137b3e: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:34.125036 4120 log.cc:824] T 00000000000000000000000000000000 P 5eab9167a5d8474698754966a3137b3e: Log is configured to *not* fsync() on all Append() calls
I20260811 18:57:34.127970 4120 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5eab9167a5d8474698754966a3137b3e: No bootstrap required, opened a new log
I20260811 18:57:34.128808 4124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:34.108624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49250 (local address 127.29.112.190:40737)
0811 18:57:34.109469 (+ 845us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:34.109481 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:34.109521 (+ 40us) server_negotiation.cc:407] Connection header received
0811 18:57:34.111309 (+ 1788us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:34.111328 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:34.111664 (+ 336us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:34.112057 (+ 393us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:34.113505 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:34.114912 (+ 1407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:34.116106 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:34.116768 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:34.116963 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:34.121344 (+ 4381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:34.121351 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:34.121363 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:34.121401 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:34.124671 (+ 3270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:34.125448 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:34.125454 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:34.125459 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:34.125684 (+ 225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:34.126081 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:34.126086 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:34.126091 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:34.126659 (+ 568us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:34.126938 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:34.127377 (+ 439us) server_negotiation.cc:299] Negotiation successful
0811 18:57:34.127786 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":621,"thread_start_us":501,"threads_started":1}
I20260811 18:57:34.137149 4120 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5eab9167a5d8474698754966a3137b3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5eab9167a5d8474698754966a3137b3e" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 40737 } }
I20260811 18:57:34.137528 4120 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5eab9167a5d8474698754966a3137b3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:34.137667 4120 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5eab9167a5d8474698754966a3137b3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5eab9167a5d8474698754966a3137b3e, State: Initialized, Role: FOLLOWER
I20260811 18:57:34.138216 4120 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5eab9167a5d8474698754966a3137b3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5eab9167a5d8474698754966a3137b3e" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 40737 } }
I20260811 18:57:34.138456 4120 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5eab9167a5d8474698754966a3137b3e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:57:34.138633 4120 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5eab9167a5d8474698754966a3137b3e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:57:34.138864 4120 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5eab9167a5d8474698754966a3137b3e [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:34.141110 4120 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5eab9167a5d8474698754966a3137b3e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5eab9167a5d8474698754966a3137b3e" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 40737 } }
I20260811 18:57:34.141633 4120 leader_election.cc:302] T 00000000000000000000000000000000 P 5eab9167a5d8474698754966a3137b3e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5eab9167a5d8474698754966a3137b3e; no voters:
I20260811 18:57:34.142300 4120 leader_election.cc:288] T 00000000000000000000000000000000 P 5eab9167a5d8474698754966a3137b3e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:57:34.142450 4127 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5eab9167a5d8474698754966a3137b3e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:34.143234 4127 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5eab9167a5d8474698754966a3137b3e [term 1 LEADER]: Becoming Leader. State: Replica: 5eab9167a5d8474698754966a3137b3e, State: Running, Role: LEADER
I20260811 18:57:34.143808 4127 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5eab9167a5d8474698754966a3137b3e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5eab9167a5d8474698754966a3137b3e" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 40737 } }
I20260811 18:57:34.144583 4120 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5eab9167a5d8474698754966a3137b3e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:57:34.146565 4130 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5eab9167a5d8474698754966a3137b3e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5eab9167a5d8474698754966a3137b3e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5eab9167a5d8474698754966a3137b3e" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 40737 } } }
I20260811 18:57:34.147058 4130 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5eab9167a5d8474698754966a3137b3e [sys.catalog]: This master's current role is: LEADER
I20260811 18:57:34.147688 4131 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5eab9167a5d8474698754966a3137b3e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5eab9167a5d8474698754966a3137b3e. Latest consensus state: current_term: 1 leader_uuid: "5eab9167a5d8474698754966a3137b3e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5eab9167a5d8474698754966a3137b3e" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 40737 } } }
I20260811 18:57:34.148012 4131 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5eab9167a5d8474698754966a3137b3e [sys.catalog]: This master's current role is: LEADER
I20260811 18:57:34.148706 4136 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:57:34.153694 4136 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:57:34.162202 4136 catalog_manager.cc:1347] Generated new cluster ID: a8bdf97bc38847d1b1ad328fe905196d
I20260811 18:57:34.162318 4136 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:57:34.186743 4136 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:57:34.188140 4136 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:57:34.194772 4136 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5eab9167a5d8474698754966a3137b3e: Generated new TSK 0
I20260811 18:57:34.196038 4136 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.112.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.129.keytab.
Entry for principal kudu/127.29.112.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.129.keytab.
I20260811 18:57:34.270290 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474615205622-30146-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474615205622-30146-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474615205622-30146-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474615205622-30146-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.129:0
--local_ip_for_outbound_sockets=127.29.112.129
--webserver_interface=127.29.112.129
--webserver_port=0
--tserver_master_addrs=127.29.112.190:40737
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.129.keytab
--principal=kudu/127.29.112.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:41957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:57:34.657621 4158 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:57:34.657984 4158 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:57:34.658159 4158 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:57:34.667909 4158 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:57:34.668112 4158 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:57:34.668298 4158 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.129
I20260811 18:57:34.679306 4158 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:41957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474615205622-30146-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474615205622-30146-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.129.keytab
--principal=kudu/127.29.112.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474615205622-30146-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.112.129
--webserver_port=0
--tserver_master_addrs=127.29.112.190:40737
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.129
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474615205622-30146-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:57:34.680735 4158 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:57:34.682108 4158 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:57:34.697031 4165 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:34.697062 4169 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:34.697180 4172 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:57:34.698303 4158 server_base.cc:1034] running on GCE node
I20260811 18:57:34.699004 4158 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:57:34.700067 4158 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:57:34.701385 4158 hybrid_clock.cc:648] HybridClock initialized: now 1786474654701307 us; error 65 us; skew 500 ppm
I20260811 18:57:34.701740 4158 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:57:34 dist-test-slave-h6n0 krb5kdc[3983](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474654, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:57:34.705159 4158 init.cc:377] Logged in from keytab as kudu/127.29.112.129@KRBTEST.COM (short username kudu)
I20260811 18:57:34.706753 4158 webserver.cc:460] Webserver started at http://127.29.112.129:38867/ using document root <none> and password file <none>
I20260811 18:57:34.707444 4158 fs_manager.cc:359] Metadata directory not provided
I20260811 18:57:34.707593 4158 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:57:34.707929 4158 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:57:34.710978 4158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474615205622-30146-0/minicluster-data/ts-0/data/instance:
uuid: "726b2ec6310646c9982d301f71f0b41f"
format_stamp: "Formatted at 2026-08-11 18:57:34 on dist-test-slave-h6n0"
server_key: "54ec53f66e73b2b342908d73b86be0f6"
server_key_iv: "a71edbe5fb455c7e2967a5256a05403b"
server_key_version: "encryptionkey@0"
I20260811 18:57:34.711773 4158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474615205622-30146-0/minicluster-data/ts-0/wal/instance:
uuid: "726b2ec6310646c9982d301f71f0b41f"
format_stamp: "Formatted at 2026-08-11 18:57:34 on dist-test-slave-h6n0"
server_key: "54ec53f66e73b2b342908d73b86be0f6"
server_key_iv: "a71edbe5fb455c7e2967a5256a05403b"
server_key_version: "encryptionkey@0"
I20260811 18:57:34.717135 4158 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.005s
I20260811 18:57:34.720691 4187 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:34.722174 4158 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260811 18:57:34.722380 4158 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474615205622-30146-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474615205622-30146-0/minicluster-data/ts-0/wal
uuid: "726b2ec6310646c9982d301f71f0b41f"
format_stamp: "Formatted at 2026-08-11 18:57:34 on dist-test-slave-h6n0"
server_key: "54ec53f66e73b2b342908d73b86be0f6"
server_key_iv: "a71edbe5fb455c7e2967a5256a05403b"
server_key_version: "encryptionkey@0"
I20260811 18:57:34.722643 4158 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474615205622-30146-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474615205622-30146-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474615205622-30146-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:57:34.747265 4158 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:57:34.754647 4158 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:57:34.755029 4158 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:57:34.756498 4158 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:57:34.760864 4158 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:57:34.760995 4158 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:34.761140 4158 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:57:34.761234 4158 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:34.796734 4158 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.129:33527
I20260811 18:57:34.796829 4364 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.129:33527 every 8 connection(s)
I20260811 18:57:34.798152 4158 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474615205622-30146-0/minicluster-data/ts-0/data/info.pb
I20260811 18:57:34.805655 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 4158
I20260811 18:57:34.805945 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474615205622-30146-0/minicluster-data/ts-0/wal/instance
I20260811 18:57:34.806788 30146 external_mini_cluster.cc:1428] Setting key 7ec679dc4459989968baa7599241cadc
Aug 11 18:57:34 dist-test-slave-h6n0 krb5kdc[3983](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474654, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.129@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 18:57:34.822995 4372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:34.801093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.129:40037 (local address 127.29.112.190:40737)
0811 18:57:34.801803 (+ 710us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:34.801812 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:34.803170 (+ 1358us) server_negotiation.cc:407] Connection header received
0811 18:57:34.806288 (+ 3118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:34.806294 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:34.806403 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:34.806537 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:34.808783 (+ 2246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:34.809536 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:34.811332 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:34.811746 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:34.811887 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:34.816068 (+ 4181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:34.816073 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:34.816077 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:34.816115 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:34.819304 (+ 3189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:34.820300 (+ 996us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:34.820305 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:34.820307 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:34.820455 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:34.820989 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:34.820993 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:34.820995 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:34.821289 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:34.821433 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:34.822443 (+ 1010us) server_negotiation.cc:299] Negotiation successful
0811 18:57:34.822718 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":545,"thread_start_us":369,"threads_started":1}
I20260811 18:57:34.824465 4373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:34.801968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:40737 (local address 127.29.112.129:40037)
0811 18:57:34.802978 (+ 1010us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:34.803025 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:34.805941 (+ 2916us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:34.806841 (+ 900us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:34.806850 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:34.807870 (+ 1020us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:57:34.808610 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:34.808640 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:34.809703 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:34.809709 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:34.811180 (+ 1471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:34.811193 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:34.812017 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:34.812025 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:34.812265 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:34.813030 (+ 765us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:57:34.813059 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:57:34.815803 (+ 2744us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:57:34.819451 (+ 3648us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:34.819463 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:34.819484 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:34.820149 (+ 665us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:34.820575 (+ 426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:34.820583 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:34.820590 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:34.820850 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:34.821435 (+ 585us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:57:34.821446 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:57:34.821912 (+ 466us) client_negotiation.cc:769] Sending connection context
0811 18:57:34.822295 (+ 383us) client_negotiation.cc:240] Negotiation successful
0811 18:57:34.822925 (+ 630us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":724,"thread_start_us":368,"threads_started":1}
I20260811 18:57:34.826167 4365 heartbeater.cc:347] Connected to a master server at 127.29.112.190:40737
I20260811 18:57:34.826563 4365 heartbeater.cc:464] Registering TS with master...
I20260811 18:57:34.827502 4365 heartbeater.cc:511] Master 127.29.112.190:40737 requested a full tablet report, sending...
I20260811 18:57:34.829923 4056 ts_manager.cc:194] Registered new tserver with Master: 726b2ec6310646c9982d301f71f0b41f (127.29.112.129:33527)
I20260811 18:57:34.831480 4056 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.129@KRBTEST.COM'} at 127.29.112.129:40037
WARNING: no policy specified for kudu/127.29.112.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.130.keytab.
Entry for principal kudu/127.29.112.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.130.keytab.
I20260811 18:57:34.866137 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474615205622-30146-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474615205622-30146-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474615205622-30146-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474615205622-30146-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.130:0
--local_ip_for_outbound_sockets=127.29.112.130
--webserver_interface=127.29.112.130
--webserver_port=0
--tserver_master_addrs=127.29.112.190:40737
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.130.keytab
--principal=kudu/127.29.112.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:41957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:57:35.225808 4379 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:57:35.226110 4379 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:57:35.226205 4379 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:57:35.233747 4379 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:57:35.233857 4379 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:57:35.233969 4379 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.130
I20260811 18:57:35.243729 4379 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:41957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474615205622-30146-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474615205622-30146-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.130.keytab
--principal=kudu/127.29.112.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474615205622-30146-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.112.130
--webserver_port=0
--tserver_master_addrs=127.29.112.190:40737
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.130
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474615205622-30146-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:57:35.245081 4379 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:57:35.246500 4379 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:57:35.260479 4387 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:35.260555 4394 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:35.260555 4399 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:57:35.261680 4379 server_base.cc:1034] running on GCE node
I20260811 18:57:35.262331 4379 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:57:35.263317 4379 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:57:35.264595 4379 hybrid_clock.cc:648] HybridClock initialized: now 1786474655264517 us; error 66 us; skew 500 ppm
I20260811 18:57:35.264976 4379 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:57:35 dist-test-slave-h6n0 krb5kdc[3983](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474655, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:57:35.268854 4379 init.cc:377] Logged in from keytab as kudu/127.29.112.130@KRBTEST.COM (short username kudu)
I20260811 18:57:35.270403 4379 webserver.cc:460] Webserver started at http://127.29.112.130:38981/ using document root <none> and password file <none>
I20260811 18:57:35.271358 4379 fs_manager.cc:359] Metadata directory not provided
I20260811 18:57:35.271507 4379 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:57:35.271943 4379 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:57:35.276422 4379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474615205622-30146-0/minicluster-data/ts-1/data/instance:
uuid: "08b6fc5b4593428fae0e9f499324c4bf"
format_stamp: "Formatted at 2026-08-11 18:57:35 on dist-test-slave-h6n0"
server_key: "51bcec2e2a64043f33cd2288c909b2ca"
server_key_iv: "0759844cd2b4171e7f39bf6909461078"
server_key_version: "encryptionkey@0"
I20260811 18:57:35.277596 4379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474615205622-30146-0/minicluster-data/ts-1/wal/instance:
uuid: "08b6fc5b4593428fae0e9f499324c4bf"
format_stamp: "Formatted at 2026-08-11 18:57:35 on dist-test-slave-h6n0"
server_key: "51bcec2e2a64043f33cd2288c909b2ca"
server_key_iv: "0759844cd2b4171e7f39bf6909461078"
server_key_version: "encryptionkey@0"
I20260811 18:57:35.285185 4379 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.009s sys 0.000s
I20260811 18:57:35.290984 4410 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:35.292779 4379 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 18:57:35.293007 4379 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474615205622-30146-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474615205622-30146-0/minicluster-data/ts-1/wal
uuid: "08b6fc5b4593428fae0e9f499324c4bf"
format_stamp: "Formatted at 2026-08-11 18:57:35 on dist-test-slave-h6n0"
server_key: "51bcec2e2a64043f33cd2288c909b2ca"
server_key_iv: "0759844cd2b4171e7f39bf6909461078"
server_key_version: "encryptionkey@0"
I20260811 18:57:35.293237 4379 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474615205622-30146-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474615205622-30146-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474615205622-30146-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:57:35.308182 4379 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:57:35.316984 4379 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:57:35.317332 4379 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:57:35.318344 4379 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:57:35.320250 4379 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:57:35.320345 4379 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:35.320482 4379 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:57:35.320547 4379 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:35.358268 4379 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.130:32899
I20260811 18:57:35.358291 4609 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.130:32899 every 8 connection(s)
I20260811 18:57:35.359596 4379 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474615205622-30146-0/minicluster-data/ts-1/data/info.pb
I20260811 18:57:35.365061 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 4379
I20260811 18:57:35.365247 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474615205622-30146-0/minicluster-data/ts-1/wal/instance
I20260811 18:57:35.365814 30146 external_mini_cluster.cc:1428] Setting key 7b96c604004e2e1519e708a2e32398e0
Aug 11 18:57:35 dist-test-slave-h6n0 krb5kdc[3983](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474655, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.130@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 18:57:35.382416 4614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:35.362230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.130:49549 (local address 127.29.112.190:40737)
0811 18:57:35.362745 (+ 515us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:35.362752 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:35.363662 (+ 910us) server_negotiation.cc:407] Connection header received
0811 18:57:35.365534 (+ 1872us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:35.365540 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:35.365627 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:35.365747 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:35.367464 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:35.368662 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:35.371230 (+ 2568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:35.371655 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:35.371760 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:35.376002 (+ 4242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:35.376011 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:35.376019 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:35.376064 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:35.378996 (+ 2932us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:35.379991 (+ 995us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:35.379995 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:35.379998 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:35.380129 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:35.380674 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:35.380678 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:35.380680 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:35.380956 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:35.381066 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:35.381969 (+ 903us) server_negotiation.cc:299] Negotiation successful
0811 18:57:35.382139 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":236,"threads_started":1}
I20260811 18:57:35.383981 4615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:35.362767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:40737 (local address 127.29.112.130:49549)
0811 18:57:35.363493 (+ 726us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:35.363541 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:35.365228 (+ 1687us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:35.365972 (+ 744us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:35.365980 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:35.366485 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:57:35.367253 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:35.367278 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:35.368815 (+ 1537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:35.368823 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:35.371044 (+ 2221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:35.371070 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:35.371894 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:35.371905 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:35.372239 (+ 334us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:35.373008 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:57:35.373042 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:57:35.375747 (+ 2705us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:57:35.379132 (+ 3385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:35.379144 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:35.379162 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:35.379808 (+ 646us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:35.380251 (+ 443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:35.380258 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:35.380264 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:35.380530 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:35.381094 (+ 564us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:57:35.381105 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:57:35.381547 (+ 442us) client_negotiation.cc:769] Sending connection context
0811 18:57:35.381859 (+ 312us) client_negotiation.cc:240] Negotiation successful
0811 18:57:35.382443 (+ 584us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":292,"threads_started":1}
I20260811 18:57:35.385492 4611 heartbeater.cc:347] Connected to a master server at 127.29.112.190:40737
I20260811 18:57:35.385867 4611 heartbeater.cc:464] Registering TS with master...
I20260811 18:57:35.386711 4611 heartbeater.cc:511] Master 127.29.112.190:40737 requested a full tablet report, sending...
I20260811 18:57:35.388437 4056 ts_manager.cc:194] Registered new tserver with Master: 08b6fc5b4593428fae0e9f499324c4bf (127.29.112.130:32899)
I20260811 18:57:35.389273 4056 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.130@KRBTEST.COM'} at 127.29.112.130:49549
WARNING: no policy specified for kudu/127.29.112.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.131.keytab.
Entry for principal kudu/127.29.112.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.131.keytab.
I20260811 18:57:35.425386 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474615205622-30146-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474615205622-30146-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474615205622-30146-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474615205622-30146-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.131:0
--local_ip_for_outbound_sockets=127.29.112.131
--webserver_interface=127.29.112.131
--webserver_port=0
--tserver_master_addrs=127.29.112.190:40737
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.131.keytab
--principal=kudu/127.29.112.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:41957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:57:35.780994 4618 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:57:35.781315 4618 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:57:35.781502 4618 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:57:35.789481 4618 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:57:35.789641 4618 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:57:35.789829 4618 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.131
I20260811 18:57:35.803038 4618 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:41957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474615205622-30146-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474615205622-30146-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.131.keytab
--principal=kudu/127.29.112.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474615205622-30146-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.112.131
--webserver_port=0
--tserver_master_addrs=127.29.112.190:40737
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.131
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474615205622-30146-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:57:35.804885 4618 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:57:35.806185 4618 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:57:35.819531 4634 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:35.819763 4626 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:35.819733 4627 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:57:35.820967 4618 server_base.cc:1034] running on GCE node
I20260811 18:57:35.821656 4618 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:57:35.823057 4618 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:57:35.824301 4618 hybrid_clock.cc:648] HybridClock initialized: now 1786474655824257 us; error 62 us; skew 500 ppm
I20260811 18:57:35.824644 4618 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:57:35 dist-test-slave-h6n0 krb5kdc[3983](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474655, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:57:35.828192 4618 init.cc:377] Logged in from keytab as kudu/127.29.112.131@KRBTEST.COM (short username kudu)
I20260811 18:57:35.829684 4618 webserver.cc:460] Webserver started at http://127.29.112.131:44027/ using document root <none> and password file <none>
I20260811 18:57:35.830415 4618 fs_manager.cc:359] Metadata directory not provided
I20260811 18:57:35.830569 4618 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:57:35.830940 4618 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:57:35.834020 4618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474615205622-30146-0/minicluster-data/ts-2/data/instance:
uuid: "fb2003d2344443b7a2dd7e870fc9ac10"
format_stamp: "Formatted at 2026-08-11 18:57:35 on dist-test-slave-h6n0"
server_key: "ea8d0bbb3498b24e9726b58c17cd20b2"
server_key_iv: "a59b69f1122204f3efd9ae7f1d52db96"
server_key_version: "encryptionkey@0"
I20260811 18:57:35.834903 4618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474615205622-30146-0/minicluster-data/ts-2/wal/instance:
uuid: "fb2003d2344443b7a2dd7e870fc9ac10"
format_stamp: "Formatted at 2026-08-11 18:57:35 on dist-test-slave-h6n0"
server_key: "ea8d0bbb3498b24e9726b58c17cd20b2"
server_key_iv: "a59b69f1122204f3efd9ae7f1d52db96"
server_key_version: "encryptionkey@0"
I20260811 18:57:35.835256 4365 heartbeater.cc:503] Master 127.29.112.190:40737 was elected leader, sending a full tablet report...
I20260811 18:57:35.840798 4618 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 18:57:35.845443 4646 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:35.848176 4618 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260811 18:57:35.848485 4618 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474615205622-30146-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474615205622-30146-0/minicluster-data/ts-2/wal
uuid: "fb2003d2344443b7a2dd7e870fc9ac10"
format_stamp: "Formatted at 2026-08-11 18:57:35 on dist-test-slave-h6n0"
server_key: "ea8d0bbb3498b24e9726b58c17cd20b2"
server_key_iv: "a59b69f1122204f3efd9ae7f1d52db96"
server_key_version: "encryptionkey@0"
I20260811 18:57:35.848794 4618 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474615205622-30146-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474615205622-30146-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474615205622-30146-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:57:35.868463 4618 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:57:35.873266 4618 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:57:35.873539 4618 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:57:35.874701 4618 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:57:35.876824 4618 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:57:35.876937 4618 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:35.877094 4618 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:57:35.877164 4618 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:35.915431 4618 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.131:45913
I20260811 18:57:35.915563 4852 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.131:45913 every 8 connection(s)
I20260811 18:57:35.916857 4618 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474615205622-30146-0/minicluster-data/ts-2/data/info.pb
I20260811 18:57:35.920744 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 4618
I20260811 18:57:35.920982 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474615205622-30146-0/minicluster-data/ts-2/wal/instance
I20260811 18:57:35.921552 30146 external_mini_cluster.cc:1428] Setting key c0a721911eb29864bd0c9fa63de70a98
Aug 11 18:57:35 dist-test-slave-h6n0 krb5kdc[3983](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474655, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.131@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 18:57:35.939987 4856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:35.919965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.131:45253 (local address 127.29.112.190:40737)
0811 18:57:35.920614 (+ 649us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:35.920622 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:35.921860 (+ 1238us) server_negotiation.cc:407] Connection header received
0811 18:57:35.924254 (+ 2394us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:35.924263 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:35.924392 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:35.924535 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:35.926276 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:35.927303 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:35.928849 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:35.929279 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:35.929382 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:35.933020 (+ 3638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:35.933025 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:35.933029 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:35.933063 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:35.936615 (+ 3552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:35.937545 (+ 930us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:35.937552 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:35.937556 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:35.937731 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:35.938356 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:35.938362 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:35.938365 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:35.938748 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:35.938900 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:35.939540 (+ 640us) server_negotiation.cc:299] Negotiation successful
0811 18:57:35.939698 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":490,"thread_start_us":336,"threads_started":1}
I20260811 18:57:35.940802 4858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:35.920510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:40737 (local address 127.29.112.131:45253)
0811 18:57:35.921662 (+ 1152us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:35.921722 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:35.923922 (+ 2200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:35.924806 (+ 884us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:35.924817 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:35.925519 (+ 702us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:57:35.926099 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:35.926117 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:35.927469 (+ 1352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:35.927473 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:35.928717 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:35.928729 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:35.929487 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:35.929492 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:35.929682 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:35.930370 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:57:35.930392 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:57:35.932776 (+ 2384us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:57:35.936783 (+ 4007us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:35.936791 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:35.936805 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:35.937383 (+ 578us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:35.937883 (+ 500us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:35.937889 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:35.937892 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:35.938188 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:35.938872 (+ 684us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:57:35.938878 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:57:35.939181 (+ 303us) client_negotiation.cc:769] Sending connection context
0811 18:57:35.939401 (+ 220us) client_negotiation.cc:240] Negotiation successful
0811 18:57:35.939758 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":880,"thread_start_us":506,"threads_started":1}
I20260811 18:57:35.942056 4854 heartbeater.cc:347] Connected to a master server at 127.29.112.190:40737
I20260811 18:57:35.942375 4854 heartbeater.cc:464] Registering TS with master...
I20260811 18:57:35.943151 4854 heartbeater.cc:511] Master 127.29.112.190:40737 requested a full tablet report, sending...
I20260811 18:57:35.944479 4056 ts_manager.cc:194] Registered new tserver with Master: fb2003d2344443b7a2dd7e870fc9ac10 (127.29.112.131:45913)
I20260811 18:57:35.945401 4056 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.131@KRBTEST.COM'} at 127.29.112.131:45253
I20260811 18:57:35.950909 30146 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 18:57:35 dist-test-slave-h6n0 krb5kdc[3983](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474655, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 18:57:36 dist-test-slave-h6n0 krb5kdc[3983](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474655, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.129@KRBTEST.COM
I20260811 18:57:36.015991 4867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:35.995078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40244 (local address 127.29.112.129:33527)
0811 18:57:35.995952 (+ 874us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:35.995960 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:35.995992 (+ 32us) server_negotiation.cc:407] Connection header received
0811 18:57:35.996065 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:35.996073 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:35.996304 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:35.996460 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:35.998676 (+ 2216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:35.999917 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:36.001932 (+ 2015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:36.002694 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:36.002854 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:36.008243 (+ 5389us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:36.008264 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:36.008284 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:36.008340 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:36.012582 (+ 4242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:36.013465 (+ 883us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:36.013478 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:36.013482 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:36.013723 (+ 241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:36.014153 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:36.014157 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:36.014159 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:36.014620 (+ 461us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:36.014889 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:36.015234 (+ 345us) server_negotiation.cc:299] Negotiation successful
0811 18:57:36.015547 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":690,"thread_start_us":477,"threads_started":1}
Aug 11 18:57:36 dist-test-slave-h6n0 krb5kdc[3983](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474655, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 18:57:36.039633 4856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:36.021763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49256 (local address 127.29.112.190:40737)
0811 18:57:36.021994 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:36.022000 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:36.022114 (+ 114us) server_negotiation.cc:407] Connection header received
0811 18:57:36.022364 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:36.022368 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:36.022459 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:36.022661 (+ 202us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:36.024114 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:36.025251 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:36.026877 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:36.027582 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:36.027704 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:36.032561 (+ 4857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:36.032569 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:36.032573 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:36.032622 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:36.035960 (+ 3338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:36.037066 (+ 1106us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:36.037071 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:36.037073 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:36.037333 (+ 260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:36.037903 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:36.037909 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:36.037912 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:36.038417 (+ 505us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:36.038812 (+ 395us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:36.039137 (+ 325us) server_negotiation.cc:299] Negotiation successful
0811 18:57:36.039366 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
W20260811 18:57:36.040347 4056 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49256
I20260811 18:57:36.041631 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 4158
I20260811 18:57:36.057005 4867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:36.042694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40254 (local address 127.29.112.129:33527)
0811 18:57:36.042961 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:36.042968 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:36.043185 (+ 217us) server_negotiation.cc:407] Connection header received
0811 18:57:36.043432 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:36.043442 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:36.043546 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:36.043665 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:36.045118 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:36.046103 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:36.047545 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:36.047992 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:36.048145 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:36.050803 (+ 2658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:36.050808 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:36.050812 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:36.050863 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:36.053705 (+ 2842us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:36.054789 (+ 1084us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:36.054793 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:36.054795 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:36.054988 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:36.055509 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:36.055513 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:36.055515 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:36.055783 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:36.055991 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:36.056408 (+ 417us) server_negotiation.cc:299] Negotiation successful
0811 18:57:36.056593 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260811 18:57:36.057669 4356 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:57:36.154436 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 4158
I20260811 18:57:36.179298 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 4379
Aug 11 18:57:36 dist-test-slave-h6n0 krb5kdc[3983](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474655, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.130@KRBTEST.COM
I20260811 18:57:36.198226 4874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:36.180233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56316 (local address 127.29.112.130:32899)
0811 18:57:36.180854 (+ 621us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:36.180861 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:36.180887 (+ 26us) server_negotiation.cc:407] Connection header received
0811 18:57:36.180958 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:36.180964 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:36.181173 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:36.181304 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:36.182466 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:36.183621 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:36.185350 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:36.185915 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:36.186028 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:36.190875 (+ 4847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:36.190887 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:36.190903 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:36.190938 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:36.194437 (+ 3499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:36.195618 (+ 1181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:36.195627 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:36.195629 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:36.195793 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:36.196393 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:36.196397 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:36.196399 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:36.196775 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:36.197189 (+ 414us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:36.197572 (+ 383us) server_negotiation.cc:299] Negotiation successful
0811 18:57:36.197807 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":476,"thread_start_us":333,"threads_started":1}
I20260811 18:57:36.199606 4597 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:57:36.285554 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 4379
I20260811 18:57:36.311108 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 4618
Aug 11 18:57:36 dist-test-slave-h6n0 krb5kdc[3983](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474655, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.131@KRBTEST.COM
I20260811 18:57:36.328152 4876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:36.312178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42404 (local address 127.29.112.131:45913)
0811 18:57:36.312867 (+ 689us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:36.312874 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:36.312904 (+ 30us) server_negotiation.cc:407] Connection header received
0811 18:57:36.312964 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:36.312971 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:36.313173 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:36.313302 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:36.314676 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:36.315698 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:36.317107 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:36.317651 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:36.317751 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:36.322170 (+ 4419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:36.322182 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:36.322197 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:36.322231 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:36.325354 (+ 3123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:36.326015 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:36.326022 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:36.326024 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:36.326155 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:36.326513 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:36.326516 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:36.326518 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:36.327051 (+ 533us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:36.327304 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:36.327535 (+ 231us) server_negotiation.cc:299] Negotiation successful
0811 18:57:36.327734 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":544,"thread_start_us":387,"threads_started":1}
I20260811 18:57:36.329447 4845 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:57:36.413255 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 4618
I20260811 18:57:36.443711 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 4003
I20260811 18:57:36.445705 4111 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:57:36.559288 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 4003
2026-08-11T18:57:36Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (4588 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:57:36 dist-test-slave-h6n0 krb5kdc[4888](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:57:36 dist-test-slave-h6n0 krb5kdc[4888](info): set up 2 sockets
Aug 11 18:57:36 dist-test-slave-h6n0 krb5kdc[4888](info): commencing operation
krb5kdc: starting...
W20260811 18:57:37.946913 30146 mini_kdc.cc:121] Time spent starting KDC: real 1.331s user 0.008s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/krb5kdc/test-user.keytab.
Aug 11 18:57:38 dist-test-slave-h6n0 krb5kdc[4888](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474658, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:57:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:57:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.112.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab.
Entry for principal kudu/127.29.112.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab.
I20260811 18:57:38.224229 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.190:33749
--webserver_interface=127.29.112.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab
--principal=kudu/127.29.112.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:41231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.190:33749
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:57:38.606024 4908 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:57:38.606359 4908 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:57:38.606562 4908 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:57:38.615042 4908 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:57:38.615154 4908 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:57:38.615239 4908 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:57:38.615326 4908 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:57:38.615391 4908 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:57:38.625795 4908 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:41231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.112.190:33749
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab
--principal=kudu/127.29.112.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.190:33749
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.112.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:57:38.627120 4908 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:57:38.628420 4908 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:57:38.640636 4919 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:38.640699 4922 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:38.640734 4918 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:57:38.642787 4908 server_base.cc:1034] running on GCE node
I20260811 18:57:38.643688 4908 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:57:38.646163 4908 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:57:38.647408 4908 hybrid_clock.cc:648] HybridClock initialized: now 1786474658647369 us; error 33 us; skew 500 ppm
I20260811 18:57:38.647763 4908 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:57:38 dist-test-slave-h6n0 krb5kdc[4888](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474658, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:57:38.651326 4908 init.cc:377] Logged in from keytab as kudu/127.29.112.190@KRBTEST.COM (short username kudu)
I20260811 18:57:38.652601 4908 fs_manager.cc:359] Metadata directory not provided
I20260811 18:57:38.652696 4908 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:57:38.653002 4908 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:57:38.656121 4908 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/minicluster-data/master-0/data/instance:
uuid: "e9d46d750bbf4635b92d868b071ed0a2"
format_stamp: "Formatted at 2026-08-11 18:57:38 on dist-test-slave-h6n0"
server_key: "e3279f3a9e16e2cbb05aa5c8b2cda9e0"
server_key_iv: "9dcd778959a303387797a8274352930a"
server_key_version: "encryptionkey@0"
I20260811 18:57:38.656924 4908 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/minicluster-data/master-0/wal/instance:
uuid: "e9d46d750bbf4635b92d868b071ed0a2"
format_stamp: "Formatted at 2026-08-11 18:57:38 on dist-test-slave-h6n0"
server_key: "e3279f3a9e16e2cbb05aa5c8b2cda9e0"
server_key_iv: "9dcd778959a303387797a8274352930a"
server_key_version: "encryptionkey@0"
I20260811 18:57:38.662071 4908 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260811 18:57:38.666464 4930 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:38.668860 4908 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.000s sys 0.003s
I20260811 18:57:38.669245 4908 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/minicluster-data/master-0/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/minicluster-data/master-0/wal
uuid: "e9d46d750bbf4635b92d868b071ed0a2"
format_stamp: "Formatted at 2026-08-11 18:57:38 on dist-test-slave-h6n0"
server_key: "e3279f3a9e16e2cbb05aa5c8b2cda9e0"
server_key_iv: "9dcd778959a303387797a8274352930a"
server_key_version: "encryptionkey@0"
I20260811 18:57:38.669524 4908 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:57:38.682739 4908 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:57:38.690486 4908 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:57:38.690840 4908 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:57:38.714062 4908 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.190:33749
I20260811 18:57:38.714109 5025 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.190:33749 every 8 connection(s)
I20260811 18:57:38.715091 4908 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
I20260811 18:57:38.716192 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 4908
I20260811 18:57:38.716423 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/minicluster-data/master-0/wal/instance
I20260811 18:57:38.717229 30146 external_mini_cluster.cc:1428] Setting key c90db510b43cc8e19a708fe298e783ca
I20260811 18:57:38.720137 5027 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 18:57:38 dist-test-slave-h6n0 krb5kdc[4888](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474658, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 18:57:38.733807 5027 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e9d46d750bbf4635b92d868b071ed0a2: Bootstrap starting.
I20260811 18:57:38.737645 5027 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e9d46d750bbf4635b92d868b071ed0a2: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:38.738900 5027 log.cc:824] T 00000000000000000000000000000000 P e9d46d750bbf4635b92d868b071ed0a2: Log is configured to *not* fsync() on all Append() calls
I20260811 18:57:38.741886 5027 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e9d46d750bbf4635b92d868b071ed0a2: No bootstrap required, opened a new log
I20260811 18:57:38.744377 5031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:38.719595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33128 (local address 127.29.112.190:33749)
0811 18:57:38.720524 (+ 929us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:38.720536 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:38.720581 (+ 45us) server_negotiation.cc:407] Connection header received
0811 18:57:38.722371 (+ 1790us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:38.722396 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:38.722899 (+ 503us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:38.723336 (+ 437us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:38.725252 (+ 1916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:38.726891 (+ 1639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:38.728700 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:38.729424 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:38.729589 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:38.734924 (+ 5335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:38.734933 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:38.734947 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:38.734986 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:38.739641 (+ 4655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:38.740439 (+ 798us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:38.740446 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:38.740452 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:38.740641 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:38.741037 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:38.741044 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:38.741048 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:38.741583 (+ 535us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:38.741930 (+ 347us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:38.742426 (+ 496us) server_negotiation.cc:299] Negotiation successful
0811 18:57:38.742954 (+ 528us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":696,"thread_start_us":436,"threads_started":1}
I20260811 18:57:38.750612 5027 raft_consensus.cc:348] T 00000000000000000000000000000000 P e9d46d750bbf4635b92d868b071ed0a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9d46d750bbf4635b92d868b071ed0a2" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 33749 } }
I20260811 18:57:38.751006 5027 raft_consensus.cc:374] T 00000000000000000000000000000000 P e9d46d750bbf4635b92d868b071ed0a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:38.751075 5027 raft_consensus.cc:729] T 00000000000000000000000000000000 P e9d46d750bbf4635b92d868b071ed0a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9d46d750bbf4635b92d868b071ed0a2, State: Initialized, Role: FOLLOWER
I20260811 18:57:38.751570 5027 consensus_queue.cc:260] T 00000000000000000000000000000000 P e9d46d750bbf4635b92d868b071ed0a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9d46d750bbf4635b92d868b071ed0a2" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 33749 } }
I20260811 18:57:38.751806 5027 raft_consensus.cc:388] T 00000000000000000000000000000000 P e9d46d750bbf4635b92d868b071ed0a2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:57:38.752014 5027 raft_consensus.cc:482] T 00000000000000000000000000000000 P e9d46d750bbf4635b92d868b071ed0a2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:57:38.752214 5027 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e9d46d750bbf4635b92d868b071ed0a2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:38.754493 5027 raft_consensus.cc:504] T 00000000000000000000000000000000 P e9d46d750bbf4635b92d868b071ed0a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9d46d750bbf4635b92d868b071ed0a2" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 33749 } }
I20260811 18:57:38.755052 5027 leader_election.cc:302] T 00000000000000000000000000000000 P e9d46d750bbf4635b92d868b071ed0a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e9d46d750bbf4635b92d868b071ed0a2; no voters:
I20260811 18:57:38.755609 5027 leader_election.cc:288] T 00000000000000000000000000000000 P e9d46d750bbf4635b92d868b071ed0a2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:57:38.755781 5036 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e9d46d750bbf4635b92d868b071ed0a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:38.756554 5036 raft_consensus.cc:686] T 00000000000000000000000000000000 P e9d46d750bbf4635b92d868b071ed0a2 [term 1 LEADER]: Becoming Leader. State: Replica: e9d46d750bbf4635b92d868b071ed0a2, State: Running, Role: LEADER
I20260811 18:57:38.757117 5036 consensus_queue.cc:237] T 00000000000000000000000000000000 P e9d46d750bbf4635b92d868b071ed0a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9d46d750bbf4635b92d868b071ed0a2" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 33749 } }
I20260811 18:57:38.757790 5027 sys_catalog.cc:564] T 00000000000000000000000000000000 P e9d46d750bbf4635b92d868b071ed0a2 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:57:38.760624 5038 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9d46d750bbf4635b92d868b071ed0a2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e9d46d750bbf4635b92d868b071ed0a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9d46d750bbf4635b92d868b071ed0a2" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 33749 } } }
I20260811 18:57:38.761138 5038 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9d46d750bbf4635b92d868b071ed0a2 [sys.catalog]: This master's current role is: LEADER
I20260811 18:57:38.760732 5039 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9d46d750bbf4635b92d868b071ed0a2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e9d46d750bbf4635b92d868b071ed0a2. Latest consensus state: current_term: 1 leader_uuid: "e9d46d750bbf4635b92d868b071ed0a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9d46d750bbf4635b92d868b071ed0a2" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 33749 } } }
I20260811 18:57:38.761595 5039 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9d46d750bbf4635b92d868b071ed0a2 [sys.catalog]: This master's current role is: LEADER
I20260811 18:57:38.763530 5050 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:57:38.770803 5050 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:57:38.782637 5050 catalog_manager.cc:1347] Generated new cluster ID: 8cccf22ec22a416cbd7c798ab29ea906
I20260811 18:57:38.782783 5050 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:57:38.806641 5050 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:57:38.808444 5050 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:57:38.815868 5050 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e9d46d750bbf4635b92d868b071ed0a2: Generated new TSK 0
I20260811 18:57:38.816991 5050 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.112.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.129.keytab.
Entry for principal kudu/127.29.112.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.129.keytab.
I20260811 18:57:38.895694 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.129:0
--local_ip_for_outbound_sockets=127.29.112.129
--webserver_interface=127.29.112.129
--webserver_port=0
--tserver_master_addrs=127.29.112.190:33749
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.129.keytab
--principal=kudu/127.29.112.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:41231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:57:39.279394 5071 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:57:39.279677 5071 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:57:39.279841 5071 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:57:39.287189 5071 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:57:39.287294 5071 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:57:39.287514 5071 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.129
I20260811 18:57:39.297309 5071 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:41231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.129.keytab
--principal=kudu/127.29.112.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.112.129
--webserver_port=0
--tserver_master_addrs=127.29.112.190:33749
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.129
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:57:39.298750 5071 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:57:39.300077 5071 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:57:39.313261 5078 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:39.313362 5080 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:39.313364 5082 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:57:39.314391 5071 server_base.cc:1034] running on GCE node
I20260811 18:57:39.315171 5071 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:57:39.316251 5071 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:57:39.317463 5071 hybrid_clock.cc:648] HybridClock initialized: now 1786474659317402 us; error 60 us; skew 500 ppm
I20260811 18:57:39.317768 5071 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:57:39 dist-test-slave-h6n0 krb5kdc[4888](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474659, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:57:39.320994 5071 init.cc:377] Logged in from keytab as kudu/127.29.112.129@KRBTEST.COM (short username kudu)
I20260811 18:57:39.322177 5071 fs_manager.cc:359] Metadata directory not provided
I20260811 18:57:39.322321 5071 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:57:39.322752 5071 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:57:39.325839 5071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/minicluster-data/ts-0/data/instance:
uuid: "2442462925a44e119b5c4d7c6f747531"
format_stamp: "Formatted at 2026-08-11 18:57:39 on dist-test-slave-h6n0"
server_key: "3f554b344b8c334613a12914b56cd40d"
server_key_iv: "506d24b3c024731ae9af4f7bacf0804f"
server_key_version: "encryptionkey@0"
I20260811 18:57:39.326746 5071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/minicluster-data/ts-0/wal/instance:
uuid: "2442462925a44e119b5c4d7c6f747531"
format_stamp: "Formatted at 2026-08-11 18:57:39 on dist-test-slave-h6n0"
server_key: "3f554b344b8c334613a12914b56cd40d"
server_key_iv: "506d24b3c024731ae9af4f7bacf0804f"
server_key_version: "encryptionkey@0"
I20260811 18:57:39.331825 5071 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260811 18:57:39.335394 5092 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:39.336644 5071 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 18:57:39.336827 5071 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/minicluster-data/ts-0/wal
uuid: "2442462925a44e119b5c4d7c6f747531"
format_stamp: "Formatted at 2026-08-11 18:57:39 on dist-test-slave-h6n0"
server_key: "3f554b344b8c334613a12914b56cd40d"
server_key_iv: "506d24b3c024731ae9af4f7bacf0804f"
server_key_version: "encryptionkey@0"
I20260811 18:57:39.337049 5071 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:57:39.349207 5071 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:57:39.353129 5071 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:57:39.353425 5071 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:57:39.354452 5071 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:57:39.355927 5071 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:57:39.356056 5071 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:39.356191 5071 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:57:39.356281 5071 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:39.396610 5071 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.129:46495
I20260811 18:57:39.396660 5296 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.129:46495 every 8 connection(s)
I20260811 18:57:39.397511 5071 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/minicluster-data/ts-0/data/info.pb
I20260811 18:57:39.404994 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 5071
I20260811 18:57:39.405244 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/minicluster-data/ts-0/wal/instance
I20260811 18:57:39.405974 30146 external_mini_cluster.cc:1428] Setting key 157f611e61a6196c398b033e9f46fe27
Aug 11 18:57:39 dist-test-slave-h6n0 krb5kdc[4888](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474659, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.129@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 18:57:39.421691 5301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:39.400907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.129:48437 (local address 127.29.112.190:33749)
0811 18:57:39.401386 (+ 479us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:39.401390 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:39.402405 (+ 1015us) server_negotiation.cc:407] Connection header received
0811 18:57:39.404393 (+ 1988us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:39.404401 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:39.404509 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:39.404663 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:39.406419 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:39.407493 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:39.409369 (+ 1876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:39.410122 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:39.410274 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:39.414272 (+ 3998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:39.414280 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:39.414287 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:39.414341 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:39.418077 (+ 3736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:39.419140 (+ 1063us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:39.419144 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:39.419146 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:39.419275 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:39.419814 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:39.419818 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:39.419820 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:39.420137 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:39.420260 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:39.421217 (+ 957us) server_negotiation.cc:299] Negotiation successful
0811 18:57:39.421422 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":243,"threads_started":1}
I20260811 18:57:39.423207 5302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:39.401478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:33749 (local address 127.29.112.129:48437)
0811 18:57:39.402222 (+ 744us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:39.402273 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:39.404061 (+ 1788us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:39.404859 (+ 798us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:39.404868 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:39.405576 (+ 708us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:57:39.406221 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:39.406241 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:39.407669 (+ 1428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:39.407674 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:39.409186 (+ 1512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:39.409199 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:39.410373 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:39.410378 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:39.410634 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:39.411366 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:57:39.411394 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:57:39.414038 (+ 2644us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:57:39.418249 (+ 4211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:39.418262 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:39.418283 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:39.419000 (+ 717us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:39.419393 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:39.419401 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:39.419407 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:39.419653 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:39.420276 (+ 623us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:57:39.420286 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:57:39.420758 (+ 472us) client_negotiation.cc:769] Sending connection context
0811 18:57:39.421065 (+ 307us) client_negotiation.cc:240] Negotiation successful
0811 18:57:39.421688 (+ 623us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":315,"threads_started":1}
I20260811 18:57:39.425725 5299 heartbeater.cc:347] Connected to a master server at 127.29.112.190:33749
I20260811 18:57:39.426072 5299 heartbeater.cc:464] Registering TS with master...
I20260811 18:57:39.426985 5299 heartbeater.cc:511] Master 127.29.112.190:33749 requested a full tablet report, sending...
I20260811 18:57:39.429822 4960 ts_manager.cc:194] Registered new tserver with Master: 2442462925a44e119b5c4d7c6f747531 (127.29.112.129:46495)
I20260811 18:57:39.432066 4960 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.129@KRBTEST.COM'} at 127.29.112.129:48437
WARNING: no policy specified for kudu/127.29.112.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.130.keytab.
Entry for principal kudu/127.29.112.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.130.keytab.
I20260811 18:57:39.465706 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.130:0
--local_ip_for_outbound_sockets=127.29.112.130
--webserver_interface=127.29.112.130
--webserver_port=0
--tserver_master_addrs=127.29.112.190:33749
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.130.keytab
--principal=kudu/127.29.112.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:41231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:57:39.821149 5305 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:57:39.821484 5305 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:57:39.821668 5305 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:57:39.829157 5305 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:57:39.829331 5305 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:57:39.829519 5305 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.130
I20260811 18:57:39.839468 5305 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:41231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.130.keytab
--principal=kudu/127.29.112.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.112.130
--webserver_port=0
--tserver_master_addrs=127.29.112.190:33749
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.130
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:57:39.840950 5305 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:57:39.842386 5305 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:57:39.855923 5312 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:39.855885 5316 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:39.856118 5314 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:57:39.856976 5305 server_base.cc:1034] running on GCE node
I20260811 18:57:39.857787 5305 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:57:39.858986 5305 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:57:39.860284 5305 hybrid_clock.cc:648] HybridClock initialized: now 1786474659860213 us; error 62 us; skew 500 ppm
I20260811 18:57:39.860664 5305 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:57:39 dist-test-slave-h6n0 krb5kdc[4888](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474659, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:57:39.869531 5305 init.cc:377] Logged in from keytab as kudu/127.29.112.130@KRBTEST.COM (short username kudu)
I20260811 18:57:39.870913 5305 fs_manager.cc:359] Metadata directory not provided
I20260811 18:57:39.871068 5305 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:57:39.871412 5305 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:57:39.874755 5305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/minicluster-data/ts-1/data/instance:
uuid: "25dc236ee00644b9ba39fff7ba8fb476"
format_stamp: "Formatted at 2026-08-11 18:57:39 on dist-test-slave-h6n0"
server_key: "fdd40368adeea3cc2c91658ece4bdcd0"
server_key_iv: "08430f9cdf5c3c5d13a9fad4eebccdba"
server_key_version: "encryptionkey@0"
I20260811 18:57:39.875614 5305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/minicluster-data/ts-1/wal/instance:
uuid: "25dc236ee00644b9ba39fff7ba8fb476"
format_stamp: "Formatted at 2026-08-11 18:57:39 on dist-test-slave-h6n0"
server_key: "fdd40368adeea3cc2c91658ece4bdcd0"
server_key_iv: "08430f9cdf5c3c5d13a9fad4eebccdba"
server_key_version: "encryptionkey@0"
I20260811 18:57:39.881075 5305 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 18:57:39.884815 5327 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:39.886190 5305 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 18:57:39.886386 5305 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/minicluster-data/ts-1/wal
uuid: "25dc236ee00644b9ba39fff7ba8fb476"
format_stamp: "Formatted at 2026-08-11 18:57:39 on dist-test-slave-h6n0"
server_key: "fdd40368adeea3cc2c91658ece4bdcd0"
server_key_iv: "08430f9cdf5c3c5d13a9fad4eebccdba"
server_key_version: "encryptionkey@0"
I20260811 18:57:39.886689 5305 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:57:39.911672 5305 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:57:39.916217 5305 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:57:39.916515 5305 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:57:39.917560 5305 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:57:39.919049 5305 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:57:39.919175 5305 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:39.919317 5305 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:57:39.919407 5305 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:39.957829 5305 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.130:38745
I20260811 18:57:39.957880 5525 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.130:38745 every 8 connection(s)
I20260811 18:57:39.958853 5305 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/minicluster-data/ts-1/data/info.pb
I20260811 18:57:39.961371 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 5305
I20260811 18:57:39.961580 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/minicluster-data/ts-1/wal/instance
I20260811 18:57:39.962220 30146 external_mini_cluster.cc:1428] Setting key d7fe294287c489e606bb4fa4e461f6fa
Aug 11 18:57:39 dist-test-slave-h6n0 krb5kdc[4888](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474659, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.130@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 18:57:39.984617 5531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:39.961564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.130:38631 (local address 127.29.112.190:33749)
0811 18:57:39.962115 (+ 551us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:39.962122 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:39.963155 (+ 1033us) server_negotiation.cc:407] Connection header received
0811 18:57:39.965790 (+ 2635us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:39.965800 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:39.965899 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:39.966028 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:39.968665 (+ 2637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:39.969839 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:39.971690 (+ 1851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:39.972470 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:39.972633 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:39.976921 (+ 4288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:39.976926 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:39.976930 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:39.976966 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:39.980478 (+ 3512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:39.981726 (+ 1248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:39.981731 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:39.981734 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:39.981907 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:39.982494 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:39.982499 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:39.982501 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:39.982919 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:39.983054 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:39.984090 (+ 1036us) server_negotiation.cc:299] Negotiation successful
0811 18:57:39.984302 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":414,"thread_start_us":276,"threads_started":1}
I20260811 18:57:39.986204 5532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:39.962093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:33749 (local address 127.29.112.130:38631)
0811 18:57:39.962950 (+ 857us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:39.963011 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:39.965137 (+ 2126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:39.967092 (+ 1955us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:39.967110 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:39.967843 (+ 733us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:57:39.968462 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:39.968479 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:39.970047 (+ 1568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:39.970052 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:39.971403 (+ 1351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:39.971416 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:39.972729 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:39.972735 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:39.972938 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:39.974076 (+ 1138us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:57:39.974108 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:57:39.976634 (+ 2526us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:57:39.980755 (+ 4121us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:39.980768 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:39.980788 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:39.981510 (+ 722us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:39.982051 (+ 541us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:39.982058 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:39.982065 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:39.982350 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:39.983059 (+ 709us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:57:39.983069 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:57:39.983551 (+ 482us) client_negotiation.cc:769] Sending connection context
0811 18:57:39.983942 (+ 391us) client_negotiation.cc:240] Negotiation successful
0811 18:57:39.984588 (+ 646us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":581,"thread_start_us":370,"threads_started":1}
I20260811 18:57:39.987969 5527 heartbeater.cc:347] Connected to a master server at 127.29.112.190:33749
I20260811 18:57:39.988336 5527 heartbeater.cc:464] Registering TS with master...
I20260811 18:57:39.989029 5527 heartbeater.cc:511] Master 127.29.112.190:33749 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.29.112.131@KRBTEST.COM; defaulting to no policy
I20260811 18:57:39.990972 4960 ts_manager.cc:194] Registered new tserver with Master: 25dc236ee00644b9ba39fff7ba8fb476 (127.29.112.130:38745)
I20260811 18:57:39.991896 4960 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.130@KRBTEST.COM'} at 127.29.112.130:38631
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.131.keytab.
Entry for principal kudu/127.29.112.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.131.keytab.
I20260811 18:57:40.022008 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.131:0
--local_ip_for_outbound_sockets=127.29.112.131
--webserver_interface=127.29.112.131
--webserver_port=0
--tserver_master_addrs=127.29.112.190:33749
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.131.keytab
--principal=kudu/127.29.112.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:41231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:57:40.367885 5538 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:57:40.368234 5538 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:57:40.368417 5538 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:57:40.375815 5538 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:57:40.375964 5538 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:57:40.376158 5538 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.131
I20260811 18:57:40.386363 5538 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:41231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.131.keytab
--principal=kudu/127.29.112.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.112.131
--webserver_port=0
--tserver_master_addrs=127.29.112.190:33749
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.131
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:57:40.387833 5538 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:57:40.389250 5538 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:57:40.407007 5546 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:40.407151 5547 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:40.407027 5549 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:57:40.408088 5538 server_base.cc:1034] running on GCE node
I20260811 18:57:40.408749 5538 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:57:40.410722 5538 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:57:40.412068 5538 hybrid_clock.cc:648] HybridClock initialized: now 1786474660411992 us; error 66 us; skew 500 ppm
I20260811 18:57:40.412845 5538 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:57:40 dist-test-slave-h6n0 krb5kdc[4888](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474660, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:57:40.417857 5538 init.cc:377] Logged in from keytab as kudu/127.29.112.131@KRBTEST.COM (short username kudu)
I20260811 18:57:40.419674 5538 fs_manager.cc:359] Metadata directory not provided
I20260811 18:57:40.419801 5538 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:57:40.420500 5538 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:57:40.423969 5538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/minicluster-data/ts-2/data/instance:
uuid: "fb9fb421982b4eabba9a51600def82bd"
format_stamp: "Formatted at 2026-08-11 18:57:40 on dist-test-slave-h6n0"
server_key: "94f0e1fbb2fe4db11478de954bffa9a7"
server_key_iv: "33ebae4b6c5a8c408d0716818f3bac78"
server_key_version: "encryptionkey@0"
I20260811 18:57:40.424962 5538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/minicluster-data/ts-2/wal/instance:
uuid: "fb9fb421982b4eabba9a51600def82bd"
format_stamp: "Formatted at 2026-08-11 18:57:40 on dist-test-slave-h6n0"
server_key: "94f0e1fbb2fe4db11478de954bffa9a7"
server_key_iv: "33ebae4b6c5a8c408d0716818f3bac78"
server_key_version: "encryptionkey@0"
I20260811 18:57:40.431219 5538 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260811 18:57:40.435458 5299 heartbeater.cc:503] Master 127.29.112.190:33749 was elected leader, sending a full tablet report...
I20260811 18:57:40.436770 5562 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:40.438922 5538 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.005s sys 0.001s
I20260811 18:57:40.439141 5538 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/minicluster-data/ts-2/wal
uuid: "fb9fb421982b4eabba9a51600def82bd"
format_stamp: "Formatted at 2026-08-11 18:57:40 on dist-test-slave-h6n0"
server_key: "94f0e1fbb2fe4db11478de954bffa9a7"
server_key_iv: "33ebae4b6c5a8c408d0716818f3bac78"
server_key_version: "encryptionkey@0"
I20260811 18:57:40.439429 5538 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:57:40.462993 5538 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:57:40.469026 5538 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:57:40.469521 5538 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:57:40.470760 5538 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:57:40.472470 5538 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:57:40.472584 5538 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:40.472762 5538 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:57:40.472844 5538 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:40.511189 5538 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.131:40969
I20260811 18:57:40.511291 5772 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.131:40969 every 8 connection(s)
I20260811 18:57:40.512085 5538 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/minicluster-data/ts-2/data/info.pb
I20260811 18:57:40.517253 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 5538
I20260811 18:57:40.517453 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/minicluster-data/ts-2/wal/instance
I20260811 18:57:40.517953 30146 external_mini_cluster.cc:1428] Setting key bedacbd198d4679b3e52f4bf61d5838d
Aug 11 18:57:40 dist-test-slave-h6n0 krb5kdc[4888](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474660, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.131@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 18:57:40.535429 5782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:40.514732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.131:58683 (local address 127.29.112.190:33749)
0811 18:57:40.515190 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:40.515194 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:40.516029 (+ 835us) server_negotiation.cc:407] Connection header received
0811 18:57:40.517923 (+ 1894us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:40.517929 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:40.518038 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:40.518170 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:40.520335 (+ 2165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:40.521502 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:40.523217 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:40.523679 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:40.523831 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:40.527612 (+ 3781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:40.527618 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:40.527621 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:40.527656 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:40.530769 (+ 3113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:40.531866 (+ 1097us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:40.531872 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:40.531876 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:40.532017 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:40.532526 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:40.532532 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:40.532541 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:40.532946 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:40.533533 (+ 587us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:40.534745 (+ 1212us) server_negotiation.cc:299] Negotiation successful
0811 18:57:40.535016 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":227,"threads_started":1}
I20260811 18:57:40.536473 5783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:40.515209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:33749 (local address 127.29.112.131:58683)
0811 18:57:40.515870 (+ 661us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:40.515909 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:40.517650 (+ 1741us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:40.518454 (+ 804us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:40.518462 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:40.519395 (+ 933us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:57:40.520041 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:40.520069 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:40.521699 (+ 1630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:40.521704 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:40.523052 (+ 1348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:40.523066 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:40.523950 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:40.523959 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:40.524205 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:40.524968 (+ 763us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:57:40.524993 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:57:40.527365 (+ 2372us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:57:40.530926 (+ 3561us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:40.530935 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:40.530952 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:40.531701 (+ 749us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:40.532147 (+ 446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:40.532153 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:40.532157 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:40.532386 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:40.533493 (+ 1107us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:57:40.533502 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:57:40.534244 (+ 742us) client_negotiation.cc:769] Sending connection context
0811 18:57:40.534614 (+ 370us) client_negotiation.cc:240] Negotiation successful
0811 18:57:40.535221 (+ 607us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":271,"threads_started":1}
I20260811 18:57:40.538002 5776 heartbeater.cc:347] Connected to a master server at 127.29.112.190:33749
I20260811 18:57:40.538475 5776 heartbeater.cc:464] Registering TS with master...
I20260811 18:57:40.539374 5776 heartbeater.cc:511] Master 127.29.112.190:33749 requested a full tablet report, sending...
I20260811 18:57:40.540858 4960 ts_manager.cc:194] Registered new tserver with Master: fb9fb421982b4eabba9a51600def82bd (127.29.112.131:40969)
I20260811 18:57:40.541704 4960 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.131@KRBTEST.COM'} at 127.29.112.131:58683
I20260811 18:57:40.546492 30146 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 18:57:40.565815 5782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:40.551622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46740 (local address 127.29.112.190:33749)
0811 18:57:40.551867 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:40.551874 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:40.552581 (+ 707us) server_negotiation.cc:407] Connection header received
0811 18:57:40.552713 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:40.552717 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:40.552781 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:40.552882 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:40.554382 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:40.555213 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:40.556502 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:40.556949 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:40.557046 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:40.559645 (+ 2599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:40.559650 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:40.559653 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:40.559685 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:40.562307 (+ 2622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:40.563221 (+ 914us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:40.563226 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:40.563228 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:40.563360 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:40.563880 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:40.563887 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:40.563889 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:40.564604 (+ 715us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:40.564802 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:40.565219 (+ 417us) server_negotiation.cc:299] Negotiation successful
0811 18:57:40.565484 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260811 18:57:40.570628 4960 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46740:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:40.579006 4960 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:40.603785 5803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:40.593362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32984 (local address 127.29.112.130:38745)
0811 18:57:40.595055 (+ 1693us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:40.595067 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:40.595100 (+ 33us) server_negotiation.cc:407] Connection header received
0811 18:57:40.595212 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:40.595222 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:40.595493 (+ 271us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:40.595688 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:57:40.596390 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:40.598008 (+ 1618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:40.601034 (+ 3026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:40.602555 (+ 1521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:40.602731 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:40.602917 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:40.603238 (+ 321us) server_negotiation.cc:299] Negotiation successful
0811 18:57:40.603438 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1512,"thread_start_us":1397,"threads_started":1}
I20260811 18:57:40.603801 5801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:40.593088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.130:38745 (local address 127.0.0.1:32984)
0811 18:57:40.594088 (+ 1000us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:40.594159 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:40.594376 (+ 217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:40.595688 (+ 1312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:40.595695 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:40.595722 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:57:40.596150 (+ 428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:40.596165 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:40.598184 (+ 2019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:40.598192 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:40.600814 (+ 2622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:40.600832 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:40.602764 (+ 1932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:40.602771 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:40.602971 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:40.603022 (+ 51us) client_negotiation.cc:769] Sending connection context
0811 18:57:40.603172 (+ 150us) client_negotiation.cc:240] Negotiation successful
0811 18:57:40.603368 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":690,"thread_start_us":2322,"threads_started":1}
I20260811 18:57:40.606288 5802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:40.593265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.131:40969 (local address 127.0.0.1:49956)
0811 18:57:40.595377 (+ 2112us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:40.595403 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:40.595518 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:40.598566 (+ 3048us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:40.598570 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:40.598597 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:57:40.599145 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:40.599159 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:40.601067 (+ 1908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:40.601073 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:40.603178 (+ 2105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:40.603194 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:40.605391 (+ 2197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:40.605397 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:40.605570 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:40.605607 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 18:57:40.605724 (+ 117us) client_negotiation.cc:240] Negotiation successful
0811 18:57:40.605857 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1991,"mutex_wait_us":27,"thread_start_us":1841,"threads_started":1}
I20260811 18:57:40.606308 5806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:40.594474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49956 (local address 127.29.112.131:40969)
0811 18:57:40.597734 (+ 3260us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:40.597742 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:40.597770 (+ 28us) server_negotiation.cc:407] Connection header received
0811 18:57:40.597848 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:40.597865 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:40.598067 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:40.598241 (+ 174us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:57:40.599386 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:40.600919 (+ 1533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:40.603374 (+ 2455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:40.605172 (+ 1798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:40.605386 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:40.605555 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:40.605782 (+ 227us) server_negotiation.cc:299] Negotiation successful
0811 18:57:40.605950 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2283,"thread_start_us":2080,"threads_started":1}
I20260811 18:57:40.607245 5805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:40.593671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:46495 (local address 127.0.0.1:49826)
0811 18:57:40.597244 (+ 3573us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:40.597261 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:40.597398 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:40.598066 (+ 668us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:40.598071 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:40.598097 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:57:40.598660 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:40.598672 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:40.601710 (+ 3038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:40.601716 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:40.603684 (+ 1968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:40.604265 (+ 581us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:40.606438 (+ 2173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:40.606445 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:40.606642 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:40.606673 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 18:57:40.606762 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 18:57:40.606887 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3448,"mutex_wait_us":1,"thread_start_us":3300,"threads_started":1}
I20260811 18:57:40.607373 5809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:40.595111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49826 (local address 127.29.112.129:46495)
0811 18:57:40.596419 (+ 1308us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:40.596426 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:40.597524 (+ 1098us) server_negotiation.cc:407] Connection header received
0811 18:57:40.597652 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:40.597661 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:40.597883 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:40.598086 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:57:40.598836 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:40.601570 (+ 2734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:40.604459 (+ 2889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:40.605911 (+ 1452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:40.606121 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:40.606269 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:40.606875 (+ 606us) server_negotiation.cc:299] Negotiation successful
0811 18:57:40.607050 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1137,"thread_start_us":975,"threads_started":1}
I20260811 18:57:40.608193 5406 tablet_service.cc:1467] Processing CreateTablet for tablet 592be6da3b314ae0bfe2e9fdbf68e388 (DEFAULT_TABLE table=test-table [id=0d40fa96795c4af79afc354137f2cb5f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:40.609802 5406 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 592be6da3b314ae0bfe2e9fdbf68e388. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:40.610370 5647 tablet_service.cc:1467] Processing CreateTablet for tablet 592be6da3b314ae0bfe2e9fdbf68e388 (DEFAULT_TABLE table=test-table [id=0d40fa96795c4af79afc354137f2cb5f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:40.610466 5173 tablet_service.cc:1467] Processing CreateTablet for tablet 592be6da3b314ae0bfe2e9fdbf68e388 (DEFAULT_TABLE table=test-table [id=0d40fa96795c4af79afc354137f2cb5f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:40.611687 5173 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 592be6da3b314ae0bfe2e9fdbf68e388. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:40.611759 5647 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 592be6da3b314ae0bfe2e9fdbf68e388. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:40.622572 5810 tablet_bootstrap.cc:492] T 592be6da3b314ae0bfe2e9fdbf68e388 P 2442462925a44e119b5c4d7c6f747531: Bootstrap starting.
I20260811 18:57:40.624528 5815 tablet_bootstrap.cc:492] T 592be6da3b314ae0bfe2e9fdbf68e388 P 25dc236ee00644b9ba39fff7ba8fb476: Bootstrap starting.
I20260811 18:57:40.626475 5810 tablet_bootstrap.cc:654] T 592be6da3b314ae0bfe2e9fdbf68e388 P 2442462925a44e119b5c4d7c6f747531: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:40.627854 5810 log.cc:824] T 592be6da3b314ae0bfe2e9fdbf68e388 P 2442462925a44e119b5c4d7c6f747531: Log is configured to *not* fsync() on all Append() calls
I20260811 18:57:40.628043 5815 tablet_bootstrap.cc:654] T 592be6da3b314ae0bfe2e9fdbf68e388 P 25dc236ee00644b9ba39fff7ba8fb476: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:40.629724 5815 log.cc:824] T 592be6da3b314ae0bfe2e9fdbf68e388 P 25dc236ee00644b9ba39fff7ba8fb476: Log is configured to *not* fsync() on all Append() calls
I20260811 18:57:40.631553 5819 tablet_bootstrap.cc:492] T 592be6da3b314ae0bfe2e9fdbf68e388 P fb9fb421982b4eabba9a51600def82bd: Bootstrap starting.
I20260811 18:57:40.633229 5815 tablet_bootstrap.cc:492] T 592be6da3b314ae0bfe2e9fdbf68e388 P 25dc236ee00644b9ba39fff7ba8fb476: No bootstrap required, opened a new log
I20260811 18:57:40.633525 5815 ts_tablet_manager.cc:1397] T 592be6da3b314ae0bfe2e9fdbf68e388 P 25dc236ee00644b9ba39fff7ba8fb476: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260811 18:57:40.633970 5810 tablet_bootstrap.cc:492] T 592be6da3b314ae0bfe2e9fdbf68e388 P 2442462925a44e119b5c4d7c6f747531: No bootstrap required, opened a new log
I20260811 18:57:40.634284 5810 ts_tablet_manager.cc:1397] T 592be6da3b314ae0bfe2e9fdbf68e388 P 2442462925a44e119b5c4d7c6f747531: Time spent bootstrapping tablet: real 0.012s user 0.010s sys 0.000s
I20260811 18:57:40.635566 5819 tablet_bootstrap.cc:654] T 592be6da3b314ae0bfe2e9fdbf68e388 P fb9fb421982b4eabba9a51600def82bd: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:40.636796 5819 log.cc:824] T 592be6da3b314ae0bfe2e9fdbf68e388 P fb9fb421982b4eabba9a51600def82bd: Log is configured to *not* fsync() on all Append() calls
I20260811 18:57:40.640756 5819 tablet_bootstrap.cc:492] T 592be6da3b314ae0bfe2e9fdbf68e388 P fb9fb421982b4eabba9a51600def82bd: No bootstrap required, opened a new log
I20260811 18:57:40.641027 5819 ts_tablet_manager.cc:1397] T 592be6da3b314ae0bfe2e9fdbf68e388 P fb9fb421982b4eabba9a51600def82bd: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260811 18:57:40.643015 5810 raft_consensus.cc:348] T 592be6da3b314ae0bfe2e9fdbf68e388 P 2442462925a44e119b5c4d7c6f747531 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25dc236ee00644b9ba39fff7ba8fb476" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 38745 } } peers { permanent_uuid: "fb9fb421982b4eabba9a51600def82bd" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 40969 } } peers { permanent_uuid: "2442462925a44e119b5c4d7c6f747531" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 46495 } }
I20260811 18:57:40.643479 5810 raft_consensus.cc:374] T 592be6da3b314ae0bfe2e9fdbf68e388 P 2442462925a44e119b5c4d7c6f747531 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:40.643607 5810 raft_consensus.cc:729] T 592be6da3b314ae0bfe2e9fdbf68e388 P 2442462925a44e119b5c4d7c6f747531 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2442462925a44e119b5c4d7c6f747531, State: Initialized, Role: FOLLOWER
I20260811 18:57:40.644258 5810 consensus_queue.cc:260] T 592be6da3b314ae0bfe2e9fdbf68e388 P 2442462925a44e119b5c4d7c6f747531 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25dc236ee00644b9ba39fff7ba8fb476" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 38745 } } peers { permanent_uuid: "fb9fb421982b4eabba9a51600def82bd" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 40969 } } peers { permanent_uuid: "2442462925a44e119b5c4d7c6f747531" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 46495 } }
I20260811 18:57:40.644731 5815 raft_consensus.cc:348] T 592be6da3b314ae0bfe2e9fdbf68e388 P 25dc236ee00644b9ba39fff7ba8fb476 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25dc236ee00644b9ba39fff7ba8fb476" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 38745 } } peers { permanent_uuid: "fb9fb421982b4eabba9a51600def82bd" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 40969 } } peers { permanent_uuid: "2442462925a44e119b5c4d7c6f747531" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 46495 } }
I20260811 18:57:40.645157 5815 raft_consensus.cc:374] T 592be6da3b314ae0bfe2e9fdbf68e388 P 25dc236ee00644b9ba39fff7ba8fb476 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:40.645260 5815 raft_consensus.cc:729] T 592be6da3b314ae0bfe2e9fdbf68e388 P 25dc236ee00644b9ba39fff7ba8fb476 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25dc236ee00644b9ba39fff7ba8fb476, State: Initialized, Role: FOLLOWER
I20260811 18:57:40.646158 5810 ts_tablet_manager.cc:1428] T 592be6da3b314ae0bfe2e9fdbf68e388 P 2442462925a44e119b5c4d7c6f747531: Time spent starting tablet: real 0.012s user 0.007s sys 0.004s
I20260811 18:57:40.646874 5815 consensus_queue.cc:260] T 592be6da3b314ae0bfe2e9fdbf68e388 P 25dc236ee00644b9ba39fff7ba8fb476 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25dc236ee00644b9ba39fff7ba8fb476" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 38745 } } peers { permanent_uuid: "fb9fb421982b4eabba9a51600def82bd" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 40969 } } peers { permanent_uuid: "2442462925a44e119b5c4d7c6f747531" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 46495 } }
I20260811 18:57:40.647990 5527 heartbeater.cc:503] Master 127.29.112.190:33749 was elected leader, sending a full tablet report...
I20260811 18:57:40.648814 5815 ts_tablet_manager.cc:1428] T 592be6da3b314ae0bfe2e9fdbf68e388 P 25dc236ee00644b9ba39fff7ba8fb476: Time spent starting tablet: real 0.015s user 0.011s sys 0.005s
W20260811 18:57:40.650470 5300 tablet.cc:2369] T 592be6da3b314ae0bfe2e9fdbf68e388 P 2442462925a44e119b5c4d7c6f747531: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:57:40.651530 5819 raft_consensus.cc:348] T 592be6da3b314ae0bfe2e9fdbf68e388 P fb9fb421982b4eabba9a51600def82bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25dc236ee00644b9ba39fff7ba8fb476" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 38745 } } peers { permanent_uuid: "fb9fb421982b4eabba9a51600def82bd" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 40969 } } peers { permanent_uuid: "2442462925a44e119b5c4d7c6f747531" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 46495 } }
I20260811 18:57:40.651958 5819 raft_consensus.cc:374] T 592be6da3b314ae0bfe2e9fdbf68e388 P fb9fb421982b4eabba9a51600def82bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:40.652073 5819 raft_consensus.cc:729] T 592be6da3b314ae0bfe2e9fdbf68e388 P fb9fb421982b4eabba9a51600def82bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb9fb421982b4eabba9a51600def82bd, State: Initialized, Role: FOLLOWER
I20260811 18:57:40.652675 5819 consensus_queue.cc:260] T 592be6da3b314ae0bfe2e9fdbf68e388 P fb9fb421982b4eabba9a51600def82bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25dc236ee00644b9ba39fff7ba8fb476" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 38745 } } peers { permanent_uuid: "fb9fb421982b4eabba9a51600def82bd" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 40969 } } peers { permanent_uuid: "2442462925a44e119b5c4d7c6f747531" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 46495 } }
I20260811 18:57:40.653815 5776 heartbeater.cc:503] Master 127.29.112.190:33749 was elected leader, sending a full tablet report...
I20260811 18:57:40.654407 5819 ts_tablet_manager.cc:1428] T 592be6da3b314ae0bfe2e9fdbf68e388 P fb9fb421982b4eabba9a51600def82bd: Time spent starting tablet: real 0.013s user 0.003s sys 0.011s
I20260811 18:57:40.671949 5828 raft_consensus.cc:482] T 592be6da3b314ae0bfe2e9fdbf68e388 P 2442462925a44e119b5c4d7c6f747531 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:40.672372 5828 raft_consensus.cc:504] T 592be6da3b314ae0bfe2e9fdbf68e388 P 2442462925a44e119b5c4d7c6f747531 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25dc236ee00644b9ba39fff7ba8fb476" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 38745 } } peers { permanent_uuid: "fb9fb421982b4eabba9a51600def82bd" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 40969 } } peers { permanent_uuid: "2442462925a44e119b5c4d7c6f747531" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 46495 } }
I20260811 18:57:40.674932 5828 leader_election.cc:288] T 592be6da3b314ae0bfe2e9fdbf68e388 P 2442462925a44e119b5c4d7c6f747531 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 25dc236ee00644b9ba39fff7ba8fb476 (127.29.112.130:38745), fb9fb421982b4eabba9a51600def82bd (127.29.112.131:40969)
I20260811 18:57:40.685482 5803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:40.675458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.129:41429 (local address 127.29.112.130:38745)
0811 18:57:40.675700 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:40.675707 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:40.676799 (+ 1092us) server_negotiation.cc:407] Connection header received
0811 18:57:40.676945 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:40.676951 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:40.677038 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:40.677149 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:57:40.678147 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:40.679398 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:40.682686 (+ 3288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:40.684201 (+ 1515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:40.684395 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:40.684481 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:40.685052 (+ 571us) server_negotiation.cc:299] Negotiation successful
0811 18:57:40.685198 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260811 18:57:40.685482 5806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:40.675401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.129:60567 (local address 127.29.112.131:40969)
0811 18:57:40.675661 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:40.675669 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:40.676723 (+ 1054us) server_negotiation.cc:407] Connection header received
0811 18:57:40.676971 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:40.676976 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:40.677063 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:40.677153 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:57:40.678147 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:40.679398 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:40.682686 (+ 3288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:40.684201 (+ 1515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:40.684408 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:40.684481 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:40.685052 (+ 571us) server_negotiation.cc:299] Negotiation successful
0811 18:57:40.685197 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 18:57:40.685730 5834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:40.675621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.130:38745 (local address 127.29.112.129:41429)
0811 18:57:40.676573 (+ 952us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:40.676589 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:40.676801 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:40.677227 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:40.677233 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:40.677275 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:57:40.677909 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:40.677928 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:40.679602 (+ 1674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:40.679608 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:40.682469 (+ 2861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:40.682488 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:40.684537 (+ 2049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:40.684545 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:40.684818 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:40.684878 (+ 60us) client_negotiation.cc:769] Sending connection context
0811 18:57:40.684975 (+ 97us) client_negotiation.cc:240] Negotiation successful
0811 18:57:40.685146 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":848,"mutex_wait_us":271,"spinlock_wait_cycles":41984,"thread_start_us":510,"threads_started":1}
I20260811 18:57:40.685752 5832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:40.675408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.131:40969 (local address 127.29.112.129:60567)
0811 18:57:40.676557 (+ 1149us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:40.676585 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:40.676755 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:40.677227 (+ 472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:40.677233 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:40.677286 (+ 53us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:57:40.677928 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:40.677938 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:40.679603 (+ 1665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:40.679608 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:40.682471 (+ 2863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:40.682488 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:40.684552 (+ 2064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:40.684557 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:40.684812 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:40.684868 (+ 56us) client_negotiation.cc:769] Sending connection context
0811 18:57:40.684975 (+ 107us) client_negotiation.cc:240] Negotiation successful
0811 18:57:40.685146 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":978,"mutex_wait_us":353,"spinlock_wait_cycles":4992,"thread_start_us":701,"threads_started":1}
I20260811 18:57:40.687142 5443 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "592be6da3b314ae0bfe2e9fdbf68e388" candidate_uuid: "2442462925a44e119b5c4d7c6f747531" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25dc236ee00644b9ba39fff7ba8fb476" is_pre_election: true
I20260811 18:57:40.687706 5443 raft_consensus.cc:2454] T 592be6da3b314ae0bfe2e9fdbf68e388 P 25dc236ee00644b9ba39fff7ba8fb476 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2442462925a44e119b5c4d7c6f747531 in term 0.
I20260811 18:57:40.687578 5685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "592be6da3b314ae0bfe2e9fdbf68e388" candidate_uuid: "2442462925a44e119b5c4d7c6f747531" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb9fb421982b4eabba9a51600def82bd" is_pre_election: true
I20260811 18:57:40.688318 5685 raft_consensus.cc:2454] T 592be6da3b314ae0bfe2e9fdbf68e388 P fb9fb421982b4eabba9a51600def82bd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2442462925a44e119b5c4d7c6f747531 in term 0.
I20260811 18:57:40.688463 5100 leader_election.cc:302] T 592be6da3b314ae0bfe2e9fdbf68e388 P 2442462925a44e119b5c4d7c6f747531 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2442462925a44e119b5c4d7c6f747531, 25dc236ee00644b9ba39fff7ba8fb476; no voters:
I20260811 18:57:40.688984 5828 raft_consensus.cc:2781] T 592be6da3b314ae0bfe2e9fdbf68e388 P 2442462925a44e119b5c4d7c6f747531 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:40.689142 5828 raft_consensus.cc:482] T 592be6da3b314ae0bfe2e9fdbf68e388 P 2442462925a44e119b5c4d7c6f747531 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:40.689323 5828 raft_consensus.cc:3037] T 592be6da3b314ae0bfe2e9fdbf68e388 P 2442462925a44e119b5c4d7c6f747531 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:40.691738 5828 raft_consensus.cc:504] T 592be6da3b314ae0bfe2e9fdbf68e388 P 2442462925a44e119b5c4d7c6f747531 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25dc236ee00644b9ba39fff7ba8fb476" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 38745 } } peers { permanent_uuid: "fb9fb421982b4eabba9a51600def82bd" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 40969 } } peers { permanent_uuid: "2442462925a44e119b5c4d7c6f747531" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 46495 } }
I20260811 18:57:40.692534 5828 leader_election.cc:288] T 592be6da3b314ae0bfe2e9fdbf68e388 P 2442462925a44e119b5c4d7c6f747531 [CANDIDATE]: Term 1 election: Requested vote from peers 25dc236ee00644b9ba39fff7ba8fb476 (127.29.112.130:38745), fb9fb421982b4eabba9a51600def82bd (127.29.112.131:40969)
I20260811 18:57:40.693048 5443 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "592be6da3b314ae0bfe2e9fdbf68e388" candidate_uuid: "2442462925a44e119b5c4d7c6f747531" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25dc236ee00644b9ba39fff7ba8fb476"
I20260811 18:57:40.693298 5443 raft_consensus.cc:3037] T 592be6da3b314ae0bfe2e9fdbf68e388 P 25dc236ee00644b9ba39fff7ba8fb476 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:40.693357 5685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "592be6da3b314ae0bfe2e9fdbf68e388" candidate_uuid: "2442462925a44e119b5c4d7c6f747531" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb9fb421982b4eabba9a51600def82bd"
I20260811 18:57:40.693626 5685 raft_consensus.cc:3037] T 592be6da3b314ae0bfe2e9fdbf68e388 P fb9fb421982b4eabba9a51600def82bd [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:40.696837 5443 raft_consensus.cc:2454] T 592be6da3b314ae0bfe2e9fdbf68e388 P 25dc236ee00644b9ba39fff7ba8fb476 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2442462925a44e119b5c4d7c6f747531 in term 1.
I20260811 18:57:40.696885 5685 raft_consensus.cc:2454] T 592be6da3b314ae0bfe2e9fdbf68e388 P fb9fb421982b4eabba9a51600def82bd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2442462925a44e119b5c4d7c6f747531 in term 1.
I20260811 18:57:40.697381 5100 leader_election.cc:302] T 592be6da3b314ae0bfe2e9fdbf68e388 P 2442462925a44e119b5c4d7c6f747531 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2442462925a44e119b5c4d7c6f747531, 25dc236ee00644b9ba39fff7ba8fb476; no voters:
I20260811 18:57:40.697842 5828 raft_consensus.cc:2781] T 592be6da3b314ae0bfe2e9fdbf68e388 P 2442462925a44e119b5c4d7c6f747531 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:40.698496 5828 raft_consensus.cc:686] T 592be6da3b314ae0bfe2e9fdbf68e388 P 2442462925a44e119b5c4d7c6f747531 [term 1 LEADER]: Becoming Leader. State: Replica: 2442462925a44e119b5c4d7c6f747531, State: Running, Role: LEADER
I20260811 18:57:40.699452 5828 consensus_queue.cc:237] T 592be6da3b314ae0bfe2e9fdbf68e388 P 2442462925a44e119b5c4d7c6f747531 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25dc236ee00644b9ba39fff7ba8fb476" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 38745 } } peers { permanent_uuid: "fb9fb421982b4eabba9a51600def82bd" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 40969 } } peers { permanent_uuid: "2442462925a44e119b5c4d7c6f747531" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 46495 } }
I20260811 18:57:40.705073 4960 catalog_manager.cc:5495] T 592be6da3b314ae0bfe2e9fdbf68e388 P 2442462925a44e119b5c4d7c6f747531 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2442462925a44e119b5c4d7c6f747531 (127.29.112.129). New cstate: current_term: 1 leader_uuid: "2442462925a44e119b5c4d7c6f747531" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25dc236ee00644b9ba39fff7ba8fb476" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 38745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fb9fb421982b4eabba9a51600def82bd" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 40969 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2442462925a44e119b5c4d7c6f747531" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 46495 } health_report { overall_health: HEALTHY } } }
W20260811 18:57:40.711125 5528 tablet.cc:2369] T 592be6da3b314ae0bfe2e9fdbf68e388 P 25dc236ee00644b9ba39fff7ba8fb476: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:57:40.743888 5809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:40.736343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49840 (local address 127.29.112.129:46495)
0811 18:57:40.736701 (+ 358us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:40.736707 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:40.736736 (+ 29us) server_negotiation.cc:407] Connection header received
0811 18:57:40.736888 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:40.736893 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:40.736996 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:40.737309 (+ 313us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:57:40.737927 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:40.738976 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:40.740927 (+ 1951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:40.741461 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:40.741590 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:40.742422 (+ 832us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:57:40.743151 (+ 729us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:57:40.743380 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:40.743434 (+ 54us) server_negotiation.cc:299] Negotiation successful
0811 18:57:40.743554 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165}
I20260811 18:57:40.754602 5443 raft_consensus.cc:1261] T 592be6da3b314ae0bfe2e9fdbf68e388 P 25dc236ee00644b9ba39fff7ba8fb476 [term 1 FOLLOWER]: Refusing update from remote peer 2442462925a44e119b5c4d7c6f747531: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:57:40.755646 5685 raft_consensus.cc:1261] T 592be6da3b314ae0bfe2e9fdbf68e388 P fb9fb421982b4eabba9a51600def82bd [term 1 FOLLOWER]: Refusing update from remote peer 2442462925a44e119b5c4d7c6f747531: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:57:40.756026 5835 consensus_queue.cc:1035] T 592be6da3b314ae0bfe2e9fdbf68e388 P 2442462925a44e119b5c4d7c6f747531 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25dc236ee00644b9ba39fff7ba8fb476" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 38745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260811 18:57:40.756937 5828 consensus_queue.cc:1035] T 592be6da3b314ae0bfe2e9fdbf68e388 P 2442462925a44e119b5c4d7c6f747531 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb9fb421982b4eabba9a51600def82bd" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 40969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 18:57:40.763692 5781 tablet.cc:2369] T 592be6da3b314ae0bfe2e9fdbf68e388 P fb9fb421982b4eabba9a51600def82bd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:57:40.774796 5841 mvcc.cc:205] Tried to move back new op lower bound from 7317400210435129344 to 7317400210229374976. Current Snapshot: MvccSnapshot[applied={T|T < 7317400210435129344}]
I20260811 18:57:40.786688 4960 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46740:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:57:40.787029 4960 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46740:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:57:40.792129 4960 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e9d46d750bbf4635b92d868b071ed0a2: Sending DeleteTablet for 3 replicas of tablet 592be6da3b314ae0bfe2e9fdbf68e388
I20260811 18:57:40.793462 5647 tablet_service.cc:1514] Processing DeleteTablet for tablet 592be6da3b314ae0bfe2e9fdbf68e388 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:40 UTC) from {username='kudu', principal='kudu/127.29.112.190@KRBTEST.COM'} at 127.0.0.1:49956
I20260811 18:57:40.793493 5406 tablet_service.cc:1514] Processing DeleteTablet for tablet 592be6da3b314ae0bfe2e9fdbf68e388 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:40 UTC) from {username='kudu', principal='kudu/127.29.112.190@KRBTEST.COM'} at 127.0.0.1:32984
I20260811 18:57:40.793844 5173 tablet_service.cc:1514] Processing DeleteTablet for tablet 592be6da3b314ae0bfe2e9fdbf68e388 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:40 UTC) from {username='kudu', principal='kudu/127.29.112.190@KRBTEST.COM'} at 127.0.0.1:49826
I20260811 18:57:40.795171 5857 tablet_replica.cc:330] T 592be6da3b314ae0bfe2e9fdbf68e388 P fb9fb421982b4eabba9a51600def82bd: stopping tablet replica
I20260811 18:57:40.795733 5857 raft_consensus.cc:2229] T 592be6da3b314ae0bfe2e9fdbf68e388 P fb9fb421982b4eabba9a51600def82bd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:40.795825 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 5071
I20260811 18:57:40.796416 5857 raft_consensus.cc:2258] T 592be6da3b314ae0bfe2e9fdbf68e388 P fb9fb421982b4eabba9a51600def82bd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:40.797411 5858 tablet_replica.cc:330] T 592be6da3b314ae0bfe2e9fdbf68e388 P 2442462925a44e119b5c4d7c6f747531: stopping tablet replica
I20260811 18:57:40.798476 5855 tablet_replica.cc:330] T 592be6da3b314ae0bfe2e9fdbf68e388 P 25dc236ee00644b9ba39fff7ba8fb476: stopping tablet replica
I20260811 18:57:40.799052 5858 raft_consensus.cc:2229] T 592be6da3b314ae0bfe2e9fdbf68e388 P 2442462925a44e119b5c4d7c6f747531 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:40.799129 5857 ts_tablet_manager.cc:1905] T 592be6da3b314ae0bfe2e9fdbf68e388 P fb9fb421982b4eabba9a51600def82bd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:40.799624 5855 raft_consensus.cc:2229] T 592be6da3b314ae0bfe2e9fdbf68e388 P 25dc236ee00644b9ba39fff7ba8fb476 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:40.800601 5858 raft_consensus.cc:2258] T 592be6da3b314ae0bfe2e9fdbf68e388 P 2442462925a44e119b5c4d7c6f747531 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:40.801210 5855 raft_consensus.cc:2258] T 592be6da3b314ae0bfe2e9fdbf68e388 P 25dc236ee00644b9ba39fff7ba8fb476 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:40.803797 5855 ts_tablet_manager.cc:1905] T 592be6da3b314ae0bfe2e9fdbf68e388 P 25dc236ee00644b9ba39fff7ba8fb476: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:40.804627 5858 ts_tablet_manager.cc:1905] T 592be6da3b314ae0bfe2e9fdbf68e388 P 2442462925a44e119b5c4d7c6f747531: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:40.807494 5857 ts_tablet_manager.cc:1918] T 592be6da3b314ae0bfe2e9fdbf68e388 P fb9fb421982b4eabba9a51600def82bd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:57:40.807695 5857 log.cc:1196] T 592be6da3b314ae0bfe2e9fdbf68e388 P fb9fb421982b4eabba9a51600def82bd: Deleting WAL directory at /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/minicluster-data/ts-2/wal/wals/592be6da3b314ae0bfe2e9fdbf68e388
I20260811 18:57:40.808321 5857 ts_tablet_manager.cc:1939] T 592be6da3b314ae0bfe2e9fdbf68e388 P fb9fb421982b4eabba9a51600def82bd: Deleting consensus metadata
I20260811 18:57:40.810329 4932 catalog_manager.cc:4841] TS fb9fb421982b4eabba9a51600def82bd (127.29.112.131:40969): tablet 592be6da3b314ae0bfe2e9fdbf68e388 (table test-table [id=0d40fa96795c4af79afc354137f2cb5f]) successfully deleted
I20260811 18:57:40.811290 5855 ts_tablet_manager.cc:1918] T 592be6da3b314ae0bfe2e9fdbf68e388 P 25dc236ee00644b9ba39fff7ba8fb476: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:57:40.811488 5855 log.cc:1196] T 592be6da3b314ae0bfe2e9fdbf68e388 P 25dc236ee00644b9ba39fff7ba8fb476: Deleting WAL directory at /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/minicluster-data/ts-1/wal/wals/592be6da3b314ae0bfe2e9fdbf68e388
I20260811 18:57:40.812073 5855 ts_tablet_manager.cc:1939] T 592be6da3b314ae0bfe2e9fdbf68e388 P 25dc236ee00644b9ba39fff7ba8fb476: Deleting consensus metadata
I20260811 18:57:40.813987 4934 catalog_manager.cc:4841] TS 25dc236ee00644b9ba39fff7ba8fb476 (127.29.112.130:38745): tablet 592be6da3b314ae0bfe2e9fdbf68e388 (table test-table [id=0d40fa96795c4af79afc354137f2cb5f]) successfully deleted
Aug 11 18:57:40 dist-test-slave-h6n0 krb5kdc[4888](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474658, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.129@KRBTEST.COM
I20260811 18:57:40.815140 5858 ts_tablet_manager.cc:1918] T 592be6da3b314ae0bfe2e9fdbf68e388 P 2442462925a44e119b5c4d7c6f747531: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:57:40.815315 5858 log.cc:1196] T 592be6da3b314ae0bfe2e9fdbf68e388 P 2442462925a44e119b5c4d7c6f747531: Deleting WAL directory at /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474615205622-30146-0/minicluster-data/ts-0/wal/wals/592be6da3b314ae0bfe2e9fdbf68e388
I20260811 18:57:40.815914 5858 ts_tablet_manager.cc:1939] T 592be6da3b314ae0bfe2e9fdbf68e388 P 2442462925a44e119b5c4d7c6f747531: Deleting consensus metadata
I20260811 18:57:40.818388 4931 catalog_manager.cc:4841] TS 2442462925a44e119b5c4d7c6f747531 (127.29.112.129:46495): tablet 592be6da3b314ae0bfe2e9fdbf68e388 (table test-table [id=0d40fa96795c4af79afc354137f2cb5f]) successfully deleted
I20260811 18:57:40.822744 5809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:40.799126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49842 (local address 127.29.112.129:46495)
0811 18:57:40.799347 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:40.799353 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:40.799377 (+ 24us) server_negotiation.cc:407] Connection header received
0811 18:57:40.799451 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:40.799458 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:40.799546 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:40.799683 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:40.802113 (+ 2430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:40.803427 (+ 1314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:40.805946 (+ 2519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:40.806743 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:40.806852 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:40.816106 (+ 9254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:40.816121 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:40.816139 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:40.816196 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:40.819747 (+ 3551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:40.820494 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:40.820510 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:40.820512 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:40.820657 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:40.821068 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:40.821073 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:40.821076 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:40.821453 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:40.821785 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:40.822094 (+ 309us) server_negotiation.cc:299] Negotiation successful
0811 18:57:40.822348 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 18:57:40.823498 5289 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:57:40.941679 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 5071
I20260811 18:57:40.979053 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 5305
Aug 11 18:57:40 dist-test-slave-h6n0 krb5kdc[4888](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474658, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.130@KRBTEST.COM
I20260811 18:57:41.006920 5803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:40.980485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32994 (local address 127.29.112.130:38745)
0811 18:57:40.980746 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:40.980752 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:40.980817 (+ 65us) server_negotiation.cc:407] Connection header received
0811 18:57:40.981091 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:40.981099 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:40.981244 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:40.981448 (+ 204us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:40.983365 (+ 1917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:40.984968 (+ 1603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:40.987355 (+ 2387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:40.988254 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:40.988532 (+ 278us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:40.996702 (+ 8170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:40.996731 (+ 29us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:40.996763 (+ 32us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:40.996834 (+ 71us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:41.001895 (+ 5061us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:41.003215 (+ 1320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:41.003227 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:41.003235 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:41.003456 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:41.004217 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:41.004225 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:41.004230 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:41.005134 (+ 904us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:41.005628 (+ 494us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:41.006018 (+ 390us) server_negotiation.cc:299] Negotiation successful
0811 18:57:41.006389 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260811 18:57:41.008231 5517 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:57:41.107139 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 5305
I20260811 18:57:41.137970 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 5538
Aug 11 18:57:41 dist-test-slave-h6n0 krb5kdc[4888](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474658, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.131@KRBTEST.COM
I20260811 18:57:41.157070 5806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:41.139120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49962 (local address 127.29.112.131:40969)
0811 18:57:41.139311 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:41.139318 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:41.139340 (+ 22us) server_negotiation.cc:407] Connection header received
0811 18:57:41.139407 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:41.139412 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:41.139482 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:41.139587 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:41.140982 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:41.142033 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:41.144104 (+ 2071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:41.144826 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:41.144965 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:41.150233 (+ 5268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:41.150253 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:41.150270 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:41.150324 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:41.153910 (+ 3586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:41.154753 (+ 843us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:41.154765 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:41.154770 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:41.154956 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:41.155398 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:41.155402 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:41.155404 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:41.155788 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:41.156133 (+ 345us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:41.156454 (+ 321us) server_negotiation.cc:299] Negotiation successful
0811 18:57:41.156705 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 18:57:41.157786 5765 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:57:41.253983 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 5538
I20260811 18:57:41.279186 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 4908
I20260811 18:57:41.280498 5013 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:57:41.394981 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 4908
2026-08-11T18:57:41Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4834 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-11T18:57:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:57:41Z Disabled control of system clock
I20260811 18:57:41.480787 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474615205622-30146-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474615205622-30146-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474615205622-30146-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.190:45071
--webserver_interface=127.29.112.190
--webserver_port=0
--builtin_ntp_servers=127.29.112.148:41185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.190:45071
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 18:57:41.845124 5874 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:57:41.845422 5874 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:57:41.845507 5874 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:57:41.852823 5874 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:57:41.852931 5874 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:57:41.852970 5874 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:57:41.853022 5874 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:57:41.853068 5874 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:57:41.862221 5874 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:41185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474615205622-30146-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474615205622-30146-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.112.190:45071
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.190:45071
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.112.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474615205622-30146-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:57:41.863538 5874 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:57:41.865267 5874 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:57:41.877130 5881 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:41.877241 5883 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:41.877170 5885 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:57:41.878803 5874 server_base.cc:1034] running on GCE node
I20260811 18:57:41.879959 5874 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:57:41.882014 5874 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:57:41.883258 5874 hybrid_clock.cc:648] HybridClock initialized: now 1786474661883222 us; error 54 us; skew 500 ppm
I20260811 18:57:41.883638 5874 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 18:57:41.886011 5874 webserver.cc:460] Webserver started at http://127.29.112.190:45275/ using document root <none> and password file <none>
I20260811 18:57:41.886775 5874 fs_manager.cc:359] Metadata directory not provided
I20260811 18:57:41.886905 5874 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:57:41.887245 5874 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:57:41.890262 5874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474615205622-30146-0/minicluster-data/master-0/data/instance:
uuid: "c20e0a7287794287a8d7243bb617c5f5"
format_stamp: "Formatted at 2026-08-11 18:57:41 on dist-test-slave-h6n0"
server_key: "aa58de3572945882f9929fafc916faca"
server_key_iv: "bc759073fdb90eeeca63e1b0058ad774"
server_key_version: "encryptionkey@0"
I20260811 18:57:41.891185 5874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474615205622-30146-0/minicluster-data/master-0/wal/instance:
uuid: "c20e0a7287794287a8d7243bb617c5f5"
format_stamp: "Formatted at 2026-08-11 18:57:41 on dist-test-slave-h6n0"
server_key: "aa58de3572945882f9929fafc916faca"
server_key_iv: "bc759073fdb90eeeca63e1b0058ad774"
server_key_version: "encryptionkey@0"
I20260811 18:57:41.896387 5874 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260811 18:57:41.899920 5898 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:41.901620 5874 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 18:57:41.901804 5874 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474615205622-30146-0/minicluster-data/master-0/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474615205622-30146-0/minicluster-data/master-0/wal
uuid: "c20e0a7287794287a8d7243bb617c5f5"
format_stamp: "Formatted at 2026-08-11 18:57:41 on dist-test-slave-h6n0"
server_key: "aa58de3572945882f9929fafc916faca"
server_key_iv: "bc759073fdb90eeeca63e1b0058ad774"
server_key_version: "encryptionkey@0"
I20260811 18:57:41.902040 5874 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474615205622-30146-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474615205622-30146-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474615205622-30146-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:57:41.908342 5874 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:57:41.909178 5874 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:57:41.909595 5874 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:57:41.933445 5874 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.190:45071
I20260811 18:57:41.933614 5999 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.190:45071 every 8 connection(s)
I20260811 18:57:41.935066 5874 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
I20260811 18:57:41.939229 6002 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:41.944350 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 5874
I20260811 18:57:41.944600 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474615205622-30146-0/minicluster-data/master-0/wal/instance
I20260811 18:57:41.945564 30146 external_mini_cluster.cc:1428] Setting key 8072f41f58be72a8d3b8b585e33cd0e0
I20260811 18:57:41.952579 6002 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c20e0a7287794287a8d7243bb617c5f5: Bootstrap starting.
I20260811 18:57:41.955020 6005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:41.947964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59906 (local address 127.29.112.190:45071)
0811 18:57:41.949291 (+ 1327us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:41.949302 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:41.949345 (+ 43us) server_negotiation.cc:407] Connection header received
0811 18:57:41.951199 (+ 1854us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:41.951221 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:41.951621 (+ 400us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:41.951983 (+ 362us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:41.953156 (+ 1173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:41.953162 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:41.953175 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:57:41.953190 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:41.953249 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:57:41.953264 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:41.953511 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:41.953659 (+ 148us) server_negotiation.cc:299] Negotiation successful
0811 18:57:41.953897 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":881,"thread_start_us":625,"threads_started":1}
I20260811 18:57:41.955888 6002 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c20e0a7287794287a8d7243bb617c5f5: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:41.957165 6002 log.cc:824] T 00000000000000000000000000000000 P c20e0a7287794287a8d7243bb617c5f5: Log is configured to *not* fsync() on all Append() calls
I20260811 18:57:41.960129 6002 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c20e0a7287794287a8d7243bb617c5f5: No bootstrap required, opened a new log
I20260811 18:57:41.968112 6002 raft_consensus.cc:348] T 00000000000000000000000000000000 P c20e0a7287794287a8d7243bb617c5f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c20e0a7287794287a8d7243bb617c5f5" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 45071 } }
I20260811 18:57:41.968472 6002 raft_consensus.cc:374] T 00000000000000000000000000000000 P c20e0a7287794287a8d7243bb617c5f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:41.968554 6002 raft_consensus.cc:729] T 00000000000000000000000000000000 P c20e0a7287794287a8d7243bb617c5f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c20e0a7287794287a8d7243bb617c5f5, State: Initialized, Role: FOLLOWER
I20260811 18:57:41.969398 6002 consensus_queue.cc:260] T 00000000000000000000000000000000 P c20e0a7287794287a8d7243bb617c5f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c20e0a7287794287a8d7243bb617c5f5" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 45071 } }
I20260811 18:57:41.969651 6002 raft_consensus.cc:388] T 00000000000000000000000000000000 P c20e0a7287794287a8d7243bb617c5f5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:57:41.969756 6002 raft_consensus.cc:482] T 00000000000000000000000000000000 P c20e0a7287794287a8d7243bb617c5f5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:57:41.969921 6002 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c20e0a7287794287a8d7243bb617c5f5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:41.972275 6002 raft_consensus.cc:504] T 00000000000000000000000000000000 P c20e0a7287794287a8d7243bb617c5f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c20e0a7287794287a8d7243bb617c5f5" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 45071 } }
I20260811 18:57:41.972795 6002 leader_election.cc:302] T 00000000000000000000000000000000 P c20e0a7287794287a8d7243bb617c5f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c20e0a7287794287a8d7243bb617c5f5; no voters:
I20260811 18:57:41.973392 6002 leader_election.cc:288] T 00000000000000000000000000000000 P c20e0a7287794287a8d7243bb617c5f5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:57:41.973575 6010 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c20e0a7287794287a8d7243bb617c5f5 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:41.974407 6010 raft_consensus.cc:686] T 00000000000000000000000000000000 P c20e0a7287794287a8d7243bb617c5f5 [term 1 LEADER]: Becoming Leader. State: Replica: c20e0a7287794287a8d7243bb617c5f5, State: Running, Role: LEADER
I20260811 18:57:41.974979 6010 consensus_queue.cc:237] T 00000000000000000000000000000000 P c20e0a7287794287a8d7243bb617c5f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c20e0a7287794287a8d7243bb617c5f5" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 45071 } }
I20260811 18:57:41.975708 6002 sys_catalog.cc:564] T 00000000000000000000000000000000 P c20e0a7287794287a8d7243bb617c5f5 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:57:41.977937 6011 sys_catalog.cc:455] T 00000000000000000000000000000000 P c20e0a7287794287a8d7243bb617c5f5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c20e0a7287794287a8d7243bb617c5f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c20e0a7287794287a8d7243bb617c5f5" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 45071 } } }
I20260811 18:57:41.978348 6011 sys_catalog.cc:458] T 00000000000000000000000000000000 P c20e0a7287794287a8d7243bb617c5f5 [sys.catalog]: This master's current role is: LEADER
I20260811 18:57:41.978060 6012 sys_catalog.cc:455] T 00000000000000000000000000000000 P c20e0a7287794287a8d7243bb617c5f5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c20e0a7287794287a8d7243bb617c5f5. Latest consensus state: current_term: 1 leader_uuid: "c20e0a7287794287a8d7243bb617c5f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c20e0a7287794287a8d7243bb617c5f5" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 45071 } } }
I20260811 18:57:41.978585 6012 sys_catalog.cc:458] T 00000000000000000000000000000000 P c20e0a7287794287a8d7243bb617c5f5 [sys.catalog]: This master's current role is: LEADER
I20260811 18:57:41.980469 6022 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:57:41.985584 6022 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:57:41.993880 6022 catalog_manager.cc:1347] Generated new cluster ID: 2d401a0b8b454b9c8a09e258da44e003
I20260811 18:57:41.994002 6022 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:57:42.021497 6022 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:57:42.021981 6022 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:57:42.038307 6022 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c20e0a7287794287a8d7243bb617c5f5: Generated new TSK 0
I20260811 18:57:42.039309 6022 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:57:42.052275 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474615205622-30146-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474615205622-30146-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474615205622-30146-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474615205622-30146-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.129:0
--local_ip_for_outbound_sockets=127.29.112.129
--webserver_interface=127.29.112.129
--webserver_port=0
--tserver_master_addrs=127.29.112.190:45071
--builtin_ntp_servers=127.29.112.148:41185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 18:57:42.421993 6050 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:57:42.422381 6050 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:57:42.422631 6050 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:57:42.432549 6050 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:57:42.432696 6050 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:57:42.433048 6050 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.129
I20260811 18:57:42.443373 6050 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:41185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474615205622-30146-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474615205622-30146-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474615205622-30146-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.112.129
--webserver_port=0
--tserver_master_addrs=127.29.112.190:45071
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.129
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474615205622-30146-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:57:42.444819 6050 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:57:42.446866 6050 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:57:42.461822 6059 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:42.462226 6066 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:42.462270 6063 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:57:42.463472 6050 server_base.cc:1034] running on GCE node
I20260811 18:57:42.464249 6050 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:57:42.465279 6050 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:57:42.466512 6050 hybrid_clock.cc:648] HybridClock initialized: now 1786474662466468 us; error 36 us; skew 500 ppm
I20260811 18:57:42.466938 6050 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 18:57:42.469676 6050 webserver.cc:460] Webserver started at http://127.29.112.129:42649/ using document root <none> and password file <none>
I20260811 18:57:42.470417 6050 fs_manager.cc:359] Metadata directory not provided
I20260811 18:57:42.470574 6050 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:57:42.470956 6050 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:57:42.474134 6050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474615205622-30146-0/minicluster-data/ts-0/data/instance:
uuid: "ad41dbaf35834010b9b4e6ab77ed009f"
format_stamp: "Formatted at 2026-08-11 18:57:42 on dist-test-slave-h6n0"
server_key: "9ff760b7becde45638c53857447cabdb"
server_key_iv: "43a0912579981a172984afc79bb4666b"
server_key_version: "encryptionkey@0"
I20260811 18:57:42.475032 6050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474615205622-30146-0/minicluster-data/ts-0/wal/instance:
uuid: "ad41dbaf35834010b9b4e6ab77ed009f"
format_stamp: "Formatted at 2026-08-11 18:57:42 on dist-test-slave-h6n0"
server_key: "9ff760b7becde45638c53857447cabdb"
server_key_iv: "43a0912579981a172984afc79bb4666b"
server_key_version: "encryptionkey@0"
I20260811 18:57:42.481133 6050 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.005s
I20260811 18:57:42.486155 6081 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:42.487720 6050 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 18:57:42.487870 6050 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474615205622-30146-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474615205622-30146-0/minicluster-data/ts-0/wal
uuid: "ad41dbaf35834010b9b4e6ab77ed009f"
format_stamp: "Formatted at 2026-08-11 18:57:42 on dist-test-slave-h6n0"
server_key: "9ff760b7becde45638c53857447cabdb"
server_key_iv: "43a0912579981a172984afc79bb4666b"
server_key_version: "encryptionkey@0"
I20260811 18:57:42.488082 6050 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474615205622-30146-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474615205622-30146-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474615205622-30146-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:57:42.493932 6050 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:57:42.494755 6050 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:57:42.494984 6050 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:57:42.495882 6050 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:57:42.497323 6050 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:57:42.497432 6050 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:42.497509 6050 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:57:42.497558 6050 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:42.533103 6050 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.129:46061
I20260811 18:57:42.533226 6271 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.129:46061 every 8 connection(s)
I20260811 18:57:42.534344 6050 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474615205622-30146-0/minicluster-data/ts-0/data/info.pb
I20260811 18:57:42.541445 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 6050
I20260811 18:57:42.541695 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474615205622-30146-0/minicluster-data/ts-0/wal/instance
I20260811 18:57:42.542601 30146 external_mini_cluster.cc:1428] Setting key b5dd4a9d94e7ce7c12ef127d6e5681f1
I20260811 18:57:42.545084 6276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:42.537422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.129:53631 (local address 127.29.112.190:45071)
0811 18:57:42.538287 (+ 865us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:42.538296 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:42.539552 (+ 1256us) server_negotiation.cc:407] Connection header received
0811 18:57:42.542183 (+ 2631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:42.542188 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:42.542250 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:42.542359 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:42.544174 (+ 1815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:42.544180 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:42.544184 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:57:42.544203 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:42.544262 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:57:42.544278 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:42.544416 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:42.544665 (+ 249us) server_negotiation.cc:299] Negotiation successful
0811 18:57:42.544722 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":642,"thread_start_us":429,"threads_started":1}
I20260811 18:57:42.546959 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474615205622-30146-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474615205622-30146-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474615205622-30146-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474615205622-30146-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.130:0
--local_ip_for_outbound_sockets=127.29.112.130
--webserver_interface=127.29.112.130
--webserver_port=0
--tserver_master_addrs=127.29.112.190:45071
--builtin_ntp_servers=127.29.112.148:41185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260811 18:57:42.547446 6277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:42.537692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:45071 (local address 127.29.112.129:53631)
0811 18:57:42.539266 (+ 1574us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:42.539344 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:42.541769 (+ 2425us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:42.542700 (+ 931us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:42.542715 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:42.542851 (+ 136us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:57:42.543802 (+ 951us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:57:42.543805 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:57:42.543861 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:57:42.543984 (+ 123us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:57:42.544435 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:57:42.544445 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:57:42.544452 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 18:57:42.544770 (+ 318us) client_negotiation.cc:240] Negotiation successful
0811 18:57:42.545358 (+ 588us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":843,"thread_start_us":441,"threads_started":1}
I20260811 18:57:42.549829 6273 heartbeater.cc:347] Connected to a master server at 127.29.112.190:45071
I20260811 18:57:42.550396 6273 heartbeater.cc:464] Registering TS with master...
I20260811 18:57:42.551316 6273 heartbeater.cc:511] Master 127.29.112.190:45071 requested a full tablet report, sending...
I20260811 18:57:42.553880 5932 ts_manager.cc:194] Registered new tserver with Master: ad41dbaf35834010b9b4e6ab77ed009f (127.29.112.129:46061)
W20260811 18:57:42.901451 6279 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:57:42.901818 6279 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:57:42.902009 6279 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:57:42.911067 6279 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:57:42.911216 6279 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:57:42.911405 6279 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.130
I20260811 18:57:42.921022 6279 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:41185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474615205622-30146-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474615205622-30146-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474615205622-30146-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.112.130
--webserver_port=0
--tserver_master_addrs=127.29.112.190:45071
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.130
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474615205622-30146-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:57:42.923513 6279 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:57:42.925549 6279 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:57:42.939543 6287 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:42.940271 6289 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:42.940361 6300 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:57:42.940708 6279 server_base.cc:1034] running on GCE node
I20260811 18:57:42.941402 6279 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:57:42.942517 6279 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:57:42.943789 6279 hybrid_clock.cc:648] HybridClock initialized: now 1786474662943717 us; error 66 us; skew 500 ppm
I20260811 18:57:42.944147 6279 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 18:57:42.946815 6279 webserver.cc:460] Webserver started at http://127.29.112.130:42175/ using document root <none> and password file <none>
I20260811 18:57:42.947645 6279 fs_manager.cc:359] Metadata directory not provided
I20260811 18:57:42.947777 6279 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:57:42.948213 6279 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:57:42.952044 6279 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474615205622-30146-0/minicluster-data/ts-1/data/instance:
uuid: "7c7015c5b9374298b2d1035a7368b01b"
format_stamp: "Formatted at 2026-08-11 18:57:42 on dist-test-slave-h6n0"
server_key: "1efbb1d086f9afb21721cab67110015c"
server_key_iv: "f4ad93929bd097ad62fcfea3fd2f7ea2"
server_key_version: "encryptionkey@0"
I20260811 18:57:42.953034 6279 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474615205622-30146-0/minicluster-data/ts-1/wal/instance:
uuid: "7c7015c5b9374298b2d1035a7368b01b"
format_stamp: "Formatted at 2026-08-11 18:57:42 on dist-test-slave-h6n0"
server_key: "1efbb1d086f9afb21721cab67110015c"
server_key_iv: "f4ad93929bd097ad62fcfea3fd2f7ea2"
server_key_version: "encryptionkey@0"
I20260811 18:57:42.959277 6279 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 18:57:42.962857 6312 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:42.964359 6279 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 18:57:42.964501 6279 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474615205622-30146-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474615205622-30146-0/minicluster-data/ts-1/wal
uuid: "7c7015c5b9374298b2d1035a7368b01b"
format_stamp: "Formatted at 2026-08-11 18:57:42 on dist-test-slave-h6n0"
server_key: "1efbb1d086f9afb21721cab67110015c"
server_key_iv: "f4ad93929bd097ad62fcfea3fd2f7ea2"
server_key_version: "encryptionkey@0"
I20260811 18:57:42.964692 6279 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474615205622-30146-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474615205622-30146-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474615205622-30146-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:57:42.970806 6279 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:57:42.971573 6279 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:57:42.971824 6279 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:57:42.972757 6279 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:57:42.974315 6279 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:57:42.974421 6279 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:42.974510 6279 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:57:42.974634 6279 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260811 18:57:43.011292 6279 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.130:33675
I20260811 18:57:43.011350 6507 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.130:33675 every 8 connection(s)
I20260811 18:57:43.012660 6279 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474615205622-30146-0/minicluster-data/ts-1/data/info.pb
I20260811 18:57:43.014639 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 6279
I20260811 18:57:43.014847 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474615205622-30146-0/minicluster-data/ts-1/wal/instance
I20260811 18:57:43.015460 30146 external_mini_cluster.cc:1428] Setting key 34d19bfaacd385983d0be09c5b3a2b76
I20260811 18:57:43.018759 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474615205622-30146-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474615205622-30146-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474615205622-30146-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474615205622-30146-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.131:0
--local_ip_for_outbound_sockets=127.29.112.131
--webserver_interface=127.29.112.131
--webserver_port=0
--tserver_master_addrs=127.29.112.190:45071
--builtin_ntp_servers=127.29.112.148:41185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260811 18:57:43.021548 6276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:43.015549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.130:35107 (local address 127.29.112.190:45071)
0811 18:57:43.015728 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:43.015733 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:43.017277 (+ 1544us) server_negotiation.cc:407] Connection header received
0811 18:57:43.019205 (+ 1928us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:43.019211 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:43.019278 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:43.019399 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:43.020776 (+ 1377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:43.020781 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:43.020786 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:57:43.020806 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:43.020849 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:57:43.020861 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:43.021002 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:43.021199 (+ 197us) server_negotiation.cc:299] Negotiation successful
0811 18:57:43.021272 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 18:57:43.022493 6511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:43.016172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:45071 (local address 127.29.112.130:35107)
0811 18:57:43.017133 (+ 961us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:43.017169 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:43.018903 (+ 1734us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:43.019633 (+ 730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:43.019641 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:43.019738 (+ 97us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:57:43.020476 (+ 738us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:57:43.020479 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:57:43.020518 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:57:43.020604 (+ 86us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:57:43.020963 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:57:43.020977 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:57:43.020986 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 18:57:43.021287 (+ 301us) client_negotiation.cc:240] Negotiation successful
0811 18:57:43.021538 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":296,"threads_started":1}
I20260811 18:57:43.024005 6508 heartbeater.cc:347] Connected to a master server at 127.29.112.190:45071
I20260811 18:57:43.024327 6508 heartbeater.cc:464] Registering TS with master...
I20260811 18:57:43.024959 6508 heartbeater.cc:511] Master 127.29.112.190:45071 requested a full tablet report, sending...
I20260811 18:57:43.026433 5932 ts_manager.cc:194] Registered new tserver with Master: 7c7015c5b9374298b2d1035a7368b01b (127.29.112.130:33675)
W20260811 18:57:43.380766 6512 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:57:43.381111 6512 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:57:43.381280 6512 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:57:43.388753 6512 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:57:43.388902 6512 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:57:43.389098 6512 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.131
I20260811 18:57:43.398522 6512 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:41185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474615205622-30146-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474615205622-30146-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474615205622-30146-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.112.131
--webserver_port=0
--tserver_master_addrs=127.29.112.190:45071
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.131
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474615205622-30146-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:57:43.399871 6512 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:57:43.401669 6512 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:57:43.415763 6520 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:43.415803 6519 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:43.416069 6525 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:57:43.417086 6512 server_base.cc:1034] running on GCE node
I20260811 18:57:43.417896 6512 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:57:43.419085 6512 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:57:43.420382 6512 hybrid_clock.cc:648] HybridClock initialized: now 1786474663420311 us; error 46 us; skew 500 ppm
I20260811 18:57:43.420708 6512 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 18:57:43.423550 6512 webserver.cc:460] Webserver started at http://127.29.112.131:33035/ using document root <none> and password file <none>
I20260811 18:57:43.424325 6512 fs_manager.cc:359] Metadata directory not provided
I20260811 18:57:43.424458 6512 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:57:43.424760 6512 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:57:43.427778 6512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474615205622-30146-0/minicluster-data/ts-2/data/instance:
uuid: "c730379113054956917f1acf4e139a0b"
format_stamp: "Formatted at 2026-08-11 18:57:43 on dist-test-slave-h6n0"
server_key: "86f58c7ad66eff36d8bac3c1511083b8"
server_key_iv: "56980be05d741493af5e4a3c97f73d8c"
server_key_version: "encryptionkey@0"
I20260811 18:57:43.428603 6512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474615205622-30146-0/minicluster-data/ts-2/wal/instance:
uuid: "c730379113054956917f1acf4e139a0b"
format_stamp: "Formatted at 2026-08-11 18:57:43 on dist-test-slave-h6n0"
server_key: "86f58c7ad66eff36d8bac3c1511083b8"
server_key_iv: "56980be05d741493af5e4a3c97f73d8c"
server_key_version: "encryptionkey@0"
I20260811 18:57:43.434978 6512 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.003s
I20260811 18:57:43.439067 6539 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:43.440660 6512 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 18:57:43.440815 6512 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474615205622-30146-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474615205622-30146-0/minicluster-data/ts-2/wal
uuid: "c730379113054956917f1acf4e139a0b"
format_stamp: "Formatted at 2026-08-11 18:57:43 on dist-test-slave-h6n0"
server_key: "86f58c7ad66eff36d8bac3c1511083b8"
server_key_iv: "56980be05d741493af5e4a3c97f73d8c"
server_key_version: "encryptionkey@0"
I20260811 18:57:43.441092 6512 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474615205622-30146-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474615205622-30146-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474615205622-30146-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:57:43.447301 6512 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:57:43.448622 6512 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:57:43.448963 6512 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:57:43.450063 6512 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:57:43.451751 6512 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:57:43.451854 6512 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:43.451989 6512 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:57:43.452080 6512 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:43.490463 6512 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.131:35555
I20260811 18:57:43.490516 6724 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.131:35555 every 8 connection(s)
I20260811 18:57:43.492663 6512 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474615205622-30146-0/minicluster-data/ts-2/data/info.pb
I20260811 18:57:43.496083 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 6512
I20260811 18:57:43.496302 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474615205622-30146-0/minicluster-data/ts-2/wal/instance
I20260811 18:57:43.496783 30146 external_mini_cluster.cc:1428] Setting key acdfa650fc44d51cf290e9eb7b3aa992
I20260811 18:57:43.502837 6276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:43.495981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.131:54475 (local address 127.29.112.190:45071)
0811 18:57:43.496205 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:43.496211 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:43.498114 (+ 1903us) server_negotiation.cc:407] Connection header received
0811 18:57:43.500168 (+ 2054us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:43.500187 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:43.500272 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:43.500361 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:43.501966 (+ 1605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:43.501971 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:43.501975 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:57:43.501999 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:43.502052 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:57:43.502067 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:43.502230 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:43.502465 (+ 235us) server_negotiation.cc:299] Negotiation successful
0811 18:57:43.502610 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260811 18:57:43.503672 6731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:43.496607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:45071 (local address 127.29.112.131:54475)
0811 18:57:43.497745 (+ 1138us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:43.497789 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:43.499848 (+ 2059us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:43.500593 (+ 745us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:43.500600 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:43.500682 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:57:43.501615 (+ 933us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:57:43.501618 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:57:43.501656 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:57:43.501742 (+ 86us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:57:43.502221 (+ 479us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:57:43.502230 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:57:43.502235 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 18:57:43.502462 (+ 227us) client_negotiation.cc:240] Negotiation successful
0811 18:57:43.502744 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":597,"thread_start_us":363,"threads_started":1}
I20260811 18:57:43.504917 6729 heartbeater.cc:347] Connected to a master server at 127.29.112.190:45071
I20260811 18:57:43.505185 6729 heartbeater.cc:464] Registering TS with master...
I20260811 18:57:43.505786 6729 heartbeater.cc:511] Master 127.29.112.190:45071 requested a full tablet report, sending...
I20260811 18:57:43.507191 5932 ts_manager.cc:194] Registered new tserver with Master: c730379113054956917f1acf4e139a0b (127.29.112.131:35555)
I20260811 18:57:43.512667 30146 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 18:57:43.520007 6276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:43.517560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59918 (local address 127.29.112.190:45071)
0811 18:57:43.517837 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:43.517843 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:43.518264 (+ 421us) server_negotiation.cc:407] Connection header received
0811 18:57:43.518371 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:43.518375 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:43.518433 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:43.518612 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:43.519467 (+ 855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:43.519472 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:43.519475 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:57:43.519490 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:43.519542 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:57:43.519555 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:43.519660 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:43.519744 (+ 84us) server_negotiation.cc:299] Negotiation successful
0811 18:57:43.519792 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120}
I20260811 18:57:43.524994 5932 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59918:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:43.533244 5932 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:43.550951 6750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:43.545955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:46061 (local address 127.0.0.1:50030)
0811 18:57:43.547665 (+ 1710us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:43.547712 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:43.547870 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:43.548687 (+ 817us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:43.548694 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:43.548778 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:57:43.549666 (+ 888us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:57:43.549668 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:57:43.549690 (+ 22us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:57:43.549748 (+ 58us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:57:43.550194 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:57:43.550209 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:57:43.550220 (+ 11us) client_negotiation.cc:769] Sending connection context
0811 18:57:43.550375 (+ 155us) client_negotiation.cc:240] Negotiation successful
0811 18:57:43.550463 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1405,"thread_start_us":1187,"threads_started":1}
I20260811 18:57:43.552402 6749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:43.546082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50030 (local address 127.29.112.129:46061)
0811 18:57:43.547018 (+ 936us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:43.547025 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:43.547817 (+ 792us) server_negotiation.cc:407] Connection header received
0811 18:57:43.548027 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:43.548038 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:43.548310 (+ 272us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:43.548475 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:43.549910 (+ 1435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:43.549920 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:43.549938 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:57:43.549961 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:43.550026 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:57:43.550064 (+ 38us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:43.551838 (+ 1774us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:43.551913 (+ 75us) server_negotiation.cc:299] Negotiation successful
0811 18:57:43.552033 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":774,"thread_start_us":621,"threads_started":1}
I20260811 18:57:43.553865 6752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:43.546709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35428 (local address 127.29.112.130:33675)
0811 18:57:43.550568 (+ 3859us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:43.550578 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:43.550613 (+ 35us) server_negotiation.cc:407] Connection header received
0811 18:57:43.550686 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:43.550694 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:43.550912 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:43.551623 (+ 711us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:43.552954 (+ 1331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:43.552962 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:43.552975 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:57:43.552993 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:43.553050 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:57:43.553076 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:43.553216 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:43.553422 (+ 206us) server_negotiation.cc:299] Negotiation successful
0811 18:57:43.553534 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3646,"thread_start_us":3521,"threads_started":1}
I20260811 18:57:43.553948 6747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:43.545997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.130:33675 (local address 127.0.0.1:35428)
0811 18:57:43.548399 (+ 2402us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:43.548419 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:43.548503 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:43.551136 (+ 2633us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:43.551141 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:43.551162 (+ 21us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:57:43.551893 (+ 731us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:57:43.551900 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:57:43.551918 (+ 18us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:57:43.551966 (+ 48us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:57:43.553229 (+ 1263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:57:43.553237 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:57:43.553246 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 18:57:43.553413 (+ 167us) client_negotiation.cc:240] Negotiation successful
0811 18:57:43.553468 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2258,"thread_start_us":2125,"threads_started":1}
I20260811 18:57:43.554612 6754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:43.548565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.131:35555 (local address 127.0.0.1:42222)
0811 18:57:43.549910 (+ 1345us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:43.549925 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:43.550002 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:43.551701 (+ 1699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:43.551706 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:43.551732 (+ 26us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:57:43.552439 (+ 707us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:57:43.552443 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:57:43.552459 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:57:43.552503 (+ 44us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:57:43.553988 (+ 1485us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:57:43.553994 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:57:43.553999 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 18:57:43.554127 (+ 128us) client_negotiation.cc:240] Negotiation successful
0811 18:57:43.554177 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":877,"thread_start_us":1917,"threads_started":1}
I20260811 18:57:43.555943 6755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:43.549821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42222 (local address 127.29.112.131:35555)
0811 18:57:43.551188 (+ 1367us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:43.551196 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:43.551229 (+ 33us) server_negotiation.cc:407] Connection header received
0811 18:57:43.551299 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:43.551306 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:43.551501 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:43.552193 (+ 692us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:43.553725 (+ 1532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:43.553733 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:43.553746 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:57:43.553766 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:43.553820 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:57:43.553841 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:43.554777 (+ 936us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:43.555437 (+ 660us) server_negotiation.cc:299] Negotiation successful
0811 18:57:43.555542 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1172,"thread_start_us":2406,"threads_started":1}
I20260811 18:57:43.557265 6273 heartbeater.cc:503] Master 127.29.112.190:45071 was elected leader, sending a full tablet report...
I20260811 18:57:43.556723 6160 tablet_service.cc:1467] Processing CreateTablet for tablet 51dffdf5066e42cd837177ce68e3cfdc (DEFAULT_TABLE table=test-table [id=944a26431e93496fab159127d377d26e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:43.558393 6160 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 51dffdf5066e42cd837177ce68e3cfdc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:43.558318 6388 tablet_service.cc:1467] Processing CreateTablet for tablet 51dffdf5066e42cd837177ce68e3cfdc (DEFAULT_TABLE table=test-table [id=944a26431e93496fab159127d377d26e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:43.559518 6388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 51dffdf5066e42cd837177ce68e3cfdc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:43.559535 6619 tablet_service.cc:1467] Processing CreateTablet for tablet 51dffdf5066e42cd837177ce68e3cfdc (DEFAULT_TABLE table=test-table [id=944a26431e93496fab159127d377d26e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:43.561022 6619 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 51dffdf5066e42cd837177ce68e3cfdc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:43.572223 6758 tablet_bootstrap.cc:492] T 51dffdf5066e42cd837177ce68e3cfdc P 7c7015c5b9374298b2d1035a7368b01b: Bootstrap starting.
I20260811 18:57:43.575098 6760 tablet_bootstrap.cc:492] T 51dffdf5066e42cd837177ce68e3cfdc P ad41dbaf35834010b9b4e6ab77ed009f: Bootstrap starting.
I20260811 18:57:43.576071 6758 tablet_bootstrap.cc:654] T 51dffdf5066e42cd837177ce68e3cfdc P 7c7015c5b9374298b2d1035a7368b01b: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:43.577450 6758 log.cc:824] T 51dffdf5066e42cd837177ce68e3cfdc P 7c7015c5b9374298b2d1035a7368b01b: Log is configured to *not* fsync() on all Append() calls
I20260811 18:57:43.579221 6760 tablet_bootstrap.cc:654] T 51dffdf5066e42cd837177ce68e3cfdc P ad41dbaf35834010b9b4e6ab77ed009f: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:43.580000 6762 tablet_bootstrap.cc:492] T 51dffdf5066e42cd837177ce68e3cfdc P c730379113054956917f1acf4e139a0b: Bootstrap starting.
I20260811 18:57:43.580471 6760 log.cc:824] T 51dffdf5066e42cd837177ce68e3cfdc P ad41dbaf35834010b9b4e6ab77ed009f: Log is configured to *not* fsync() on all Append() calls
I20260811 18:57:43.580801 6758 tablet_bootstrap.cc:492] T 51dffdf5066e42cd837177ce68e3cfdc P 7c7015c5b9374298b2d1035a7368b01b: No bootstrap required, opened a new log
I20260811 18:57:43.581104 6758 ts_tablet_manager.cc:1397] T 51dffdf5066e42cd837177ce68e3cfdc P 7c7015c5b9374298b2d1035a7368b01b: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260811 18:57:43.583295 6762 tablet_bootstrap.cc:654] T 51dffdf5066e42cd837177ce68e3cfdc P c730379113054956917f1acf4e139a0b: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:43.583678 6760 tablet_bootstrap.cc:492] T 51dffdf5066e42cd837177ce68e3cfdc P ad41dbaf35834010b9b4e6ab77ed009f: No bootstrap required, opened a new log
I20260811 18:57:43.583948 6760 ts_tablet_manager.cc:1397] T 51dffdf5066e42cd837177ce68e3cfdc P ad41dbaf35834010b9b4e6ab77ed009f: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260811 18:57:43.584336 6762 log.cc:824] T 51dffdf5066e42cd837177ce68e3cfdc P c730379113054956917f1acf4e139a0b: Log is configured to *not* fsync() on all Append() calls
I20260811 18:57:43.587162 6762 tablet_bootstrap.cc:492] T 51dffdf5066e42cd837177ce68e3cfdc P c730379113054956917f1acf4e139a0b: No bootstrap required, opened a new log
I20260811 18:57:43.587472 6762 ts_tablet_manager.cc:1397] T 51dffdf5066e42cd837177ce68e3cfdc P c730379113054956917f1acf4e139a0b: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.003s
I20260811 18:57:43.591908 6758 raft_consensus.cc:348] T 51dffdf5066e42cd837177ce68e3cfdc P 7c7015c5b9374298b2d1035a7368b01b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad41dbaf35834010b9b4e6ab77ed009f" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 46061 } } peers { permanent_uuid: "7c7015c5b9374298b2d1035a7368b01b" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33675 } } peers { permanent_uuid: "c730379113054956917f1acf4e139a0b" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 35555 } }
I20260811 18:57:43.592403 6758 raft_consensus.cc:374] T 51dffdf5066e42cd837177ce68e3cfdc P 7c7015c5b9374298b2d1035a7368b01b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:43.592525 6758 raft_consensus.cc:729] T 51dffdf5066e42cd837177ce68e3cfdc P 7c7015c5b9374298b2d1035a7368b01b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c7015c5b9374298b2d1035a7368b01b, State: Initialized, Role: FOLLOWER
I20260811 18:57:43.593256 6758 consensus_queue.cc:260] T 51dffdf5066e42cd837177ce68e3cfdc P 7c7015c5b9374298b2d1035a7368b01b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad41dbaf35834010b9b4e6ab77ed009f" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 46061 } } peers { permanent_uuid: "7c7015c5b9374298b2d1035a7368b01b" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33675 } } peers { permanent_uuid: "c730379113054956917f1acf4e139a0b" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 35555 } }
I20260811 18:57:43.594991 6760 raft_consensus.cc:348] T 51dffdf5066e42cd837177ce68e3cfdc P ad41dbaf35834010b9b4e6ab77ed009f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad41dbaf35834010b9b4e6ab77ed009f" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 46061 } } peers { permanent_uuid: "7c7015c5b9374298b2d1035a7368b01b" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33675 } } peers { permanent_uuid: "c730379113054956917f1acf4e139a0b" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 35555 } }
I20260811 18:57:43.595407 6760 raft_consensus.cc:374] T 51dffdf5066e42cd837177ce68e3cfdc P ad41dbaf35834010b9b4e6ab77ed009f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:43.595532 6760 raft_consensus.cc:729] T 51dffdf5066e42cd837177ce68e3cfdc P ad41dbaf35834010b9b4e6ab77ed009f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad41dbaf35834010b9b4e6ab77ed009f, State: Initialized, Role: FOLLOWER
I20260811 18:57:43.596021 6758 ts_tablet_manager.cc:1428] T 51dffdf5066e42cd837177ce68e3cfdc P 7c7015c5b9374298b2d1035a7368b01b: Time spent starting tablet: real 0.015s user 0.012s sys 0.002s
I20260811 18:57:43.596199 6760 consensus_queue.cc:260] T 51dffdf5066e42cd837177ce68e3cfdc P ad41dbaf35834010b9b4e6ab77ed009f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad41dbaf35834010b9b4e6ab77ed009f" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 46061 } } peers { permanent_uuid: "7c7015c5b9374298b2d1035a7368b01b" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33675 } } peers { permanent_uuid: "c730379113054956917f1acf4e139a0b" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 35555 } }
I20260811 18:57:43.596496 6508 heartbeater.cc:503] Master 127.29.112.190:45071 was elected leader, sending a full tablet report...
I20260811 18:57:43.596526 6762 raft_consensus.cc:348] T 51dffdf5066e42cd837177ce68e3cfdc P c730379113054956917f1acf4e139a0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad41dbaf35834010b9b4e6ab77ed009f" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 46061 } } peers { permanent_uuid: "7c7015c5b9374298b2d1035a7368b01b" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33675 } } peers { permanent_uuid: "c730379113054956917f1acf4e139a0b" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 35555 } }
I20260811 18:57:43.597002 6762 raft_consensus.cc:374] T 51dffdf5066e42cd837177ce68e3cfdc P c730379113054956917f1acf4e139a0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:43.597136 6762 raft_consensus.cc:729] T 51dffdf5066e42cd837177ce68e3cfdc P c730379113054956917f1acf4e139a0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c730379113054956917f1acf4e139a0b, State: Initialized, Role: FOLLOWER
I20260811 18:57:43.597831 6762 consensus_queue.cc:260] T 51dffdf5066e42cd837177ce68e3cfdc P c730379113054956917f1acf4e139a0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad41dbaf35834010b9b4e6ab77ed009f" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 46061 } } peers { permanent_uuid: "7c7015c5b9374298b2d1035a7368b01b" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33675 } } peers { permanent_uuid: "c730379113054956917f1acf4e139a0b" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 35555 } }
I20260811 18:57:43.598318 6760 ts_tablet_manager.cc:1428] T 51dffdf5066e42cd837177ce68e3cfdc P ad41dbaf35834010b9b4e6ab77ed009f: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260811 18:57:43.599391 6729 heartbeater.cc:503] Master 127.29.112.190:45071 was elected leader, sending a full tablet report...
I20260811 18:57:43.600396 6762 ts_tablet_manager.cc:1428] T 51dffdf5066e42cd837177ce68e3cfdc P c730379113054956917f1acf4e139a0b: Time spent starting tablet: real 0.013s user 0.011s sys 0.001s
I20260811 18:57:43.677757 6768 raft_consensus.cc:482] T 51dffdf5066e42cd837177ce68e3cfdc P 7c7015c5b9374298b2d1035a7368b01b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:43.678103 6768 raft_consensus.cc:504] T 51dffdf5066e42cd837177ce68e3cfdc P 7c7015c5b9374298b2d1035a7368b01b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad41dbaf35834010b9b4e6ab77ed009f" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 46061 } } peers { permanent_uuid: "7c7015c5b9374298b2d1035a7368b01b" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33675 } } peers { permanent_uuid: "c730379113054956917f1acf4e139a0b" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 35555 } }
I20260811 18:57:43.680002 6768 leader_election.cc:288] T 51dffdf5066e42cd837177ce68e3cfdc P 7c7015c5b9374298b2d1035a7368b01b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad41dbaf35834010b9b4e6ab77ed009f (127.29.112.129:46061), c730379113054956917f1acf4e139a0b (127.29.112.131:35555)
I20260811 18:57:43.684051 6755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:43.680763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.130:52935 (local address 127.29.112.131:35555)
0811 18:57:43.680958 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:43.680968 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:43.681679 (+ 711us) server_negotiation.cc:407] Connection header received
0811 18:57:43.681912 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:43.681922 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:43.681997 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:43.682102 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:43.683295 (+ 1193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:43.683302 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:43.683307 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:57:43.683326 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:43.683360 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:57:43.683375 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:43.683514 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:43.683661 (+ 147us) server_negotiation.cc:299] Negotiation successful
0811 18:57:43.683737 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 18:57:43.684243 6749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:43.680647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.130:52799 (local address 127.29.112.129:46061)
0811 18:57:43.680856 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:43.680864 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:43.682027 (+ 1163us) server_negotiation.cc:407] Connection header received
0811 18:57:43.682132 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:43.682137 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:43.682198 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:43.682344 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:43.683540 (+ 1196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:43.683547 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:43.683552 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:57:43.683575 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:43.683627 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:57:43.683647 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:43.683785 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:43.683890 (+ 105us) server_negotiation.cc:299] Negotiation successful
0811 18:57:43.683959 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 18:57:43.684607 6775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:43.680266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.131:35555 (local address 127.29.112.130:52935)
0811 18:57:43.681388 (+ 1122us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:43.681411 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:43.681554 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:43.682159 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:43.682164 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:43.682190 (+ 26us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:57:43.683101 (+ 911us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:57:43.683103 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:57:43.683119 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:57:43.683161 (+ 42us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:57:43.683517 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:57:43.683524 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:57:43.683530 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 18:57:43.683725 (+ 195us) client_negotiation.cc:240] Negotiation successful
0811 18:57:43.683789 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":910,"thread_start_us":866,"threads_started":1}
I20260811 18:57:43.684931 6774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:43.680445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:46061 (local address 127.29.112.130:52799)
0811 18:57:43.681869 (+ 1424us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:43.681896 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:43.682008 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:43.682414 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:43.682418 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:43.682437 (+ 19us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:57:43.683293 (+ 856us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:57:43.683297 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:57:43.683313 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:57:43.683357 (+ 44us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:57:43.683761 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:57:43.683766 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:57:43.683770 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 18:57:43.683924 (+ 154us) client_negotiation.cc:240] Negotiation successful
0811 18:57:43.683987 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":996,"thread_start_us":527,"threads_started":1}
I20260811 18:57:43.685664 6653 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51dffdf5066e42cd837177ce68e3cfdc" candidate_uuid: "7c7015c5b9374298b2d1035a7368b01b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c730379113054956917f1acf4e139a0b" is_pre_election: true
I20260811 18:57:43.686235 6653 raft_consensus.cc:2454] T 51dffdf5066e42cd837177ce68e3cfdc P c730379113054956917f1acf4e139a0b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c7015c5b9374298b2d1035a7368b01b in term 0.
I20260811 18:57:43.686008 6200 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51dffdf5066e42cd837177ce68e3cfdc" candidate_uuid: "7c7015c5b9374298b2d1035a7368b01b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad41dbaf35834010b9b4e6ab77ed009f" is_pre_election: true
I20260811 18:57:43.686707 6200 raft_consensus.cc:2454] T 51dffdf5066e42cd837177ce68e3cfdc P ad41dbaf35834010b9b4e6ab77ed009f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c7015c5b9374298b2d1035a7368b01b in term 0.
I20260811 18:57:43.687004 6318 leader_election.cc:302] T 51dffdf5066e42cd837177ce68e3cfdc P 7c7015c5b9374298b2d1035a7368b01b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c7015c5b9374298b2d1035a7368b01b, c730379113054956917f1acf4e139a0b; no voters:
I20260811 18:57:43.687436 6768 raft_consensus.cc:2781] T 51dffdf5066e42cd837177ce68e3cfdc P 7c7015c5b9374298b2d1035a7368b01b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:43.687605 6768 raft_consensus.cc:482] T 51dffdf5066e42cd837177ce68e3cfdc P 7c7015c5b9374298b2d1035a7368b01b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:43.687744 6768 raft_consensus.cc:3037] T 51dffdf5066e42cd837177ce68e3cfdc P 7c7015c5b9374298b2d1035a7368b01b [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:43.690225 6768 raft_consensus.cc:504] T 51dffdf5066e42cd837177ce68e3cfdc P 7c7015c5b9374298b2d1035a7368b01b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad41dbaf35834010b9b4e6ab77ed009f" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 46061 } } peers { permanent_uuid: "7c7015c5b9374298b2d1035a7368b01b" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33675 } } peers { permanent_uuid: "c730379113054956917f1acf4e139a0b" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 35555 } }
I20260811 18:57:43.690949 6768 leader_election.cc:288] T 51dffdf5066e42cd837177ce68e3cfdc P 7c7015c5b9374298b2d1035a7368b01b [CANDIDATE]: Term 1 election: Requested vote from peers ad41dbaf35834010b9b4e6ab77ed009f (127.29.112.129:46061), c730379113054956917f1acf4e139a0b (127.29.112.131:35555)
I20260811 18:57:43.691488 6653 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51dffdf5066e42cd837177ce68e3cfdc" candidate_uuid: "7c7015c5b9374298b2d1035a7368b01b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c730379113054956917f1acf4e139a0b"
I20260811 18:57:43.691489 6200 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51dffdf5066e42cd837177ce68e3cfdc" candidate_uuid: "7c7015c5b9374298b2d1035a7368b01b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad41dbaf35834010b9b4e6ab77ed009f"
I20260811 18:57:43.691747 6200 raft_consensus.cc:3037] T 51dffdf5066e42cd837177ce68e3cfdc P ad41dbaf35834010b9b4e6ab77ed009f [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:43.691733 6653 raft_consensus.cc:3037] T 51dffdf5066e42cd837177ce68e3cfdc P c730379113054956917f1acf4e139a0b [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:43.695484 6653 raft_consensus.cc:2454] T 51dffdf5066e42cd837177ce68e3cfdc P c730379113054956917f1acf4e139a0b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c7015c5b9374298b2d1035a7368b01b in term 1.
I20260811 18:57:43.695484 6200 raft_consensus.cc:2454] T 51dffdf5066e42cd837177ce68e3cfdc P ad41dbaf35834010b9b4e6ab77ed009f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c7015c5b9374298b2d1035a7368b01b in term 1.
I20260811 18:57:43.696015 6318 leader_election.cc:302] T 51dffdf5066e42cd837177ce68e3cfdc P 7c7015c5b9374298b2d1035a7368b01b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c7015c5b9374298b2d1035a7368b01b, c730379113054956917f1acf4e139a0b; no voters:
I20260811 18:57:43.696348 6768 raft_consensus.cc:2781] T 51dffdf5066e42cd837177ce68e3cfdc P 7c7015c5b9374298b2d1035a7368b01b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:43.696924 6768 raft_consensus.cc:686] T 51dffdf5066e42cd837177ce68e3cfdc P 7c7015c5b9374298b2d1035a7368b01b [term 1 LEADER]: Becoming Leader. State: Replica: 7c7015c5b9374298b2d1035a7368b01b, State: Running, Role: LEADER
I20260811 18:57:43.697582 6768 consensus_queue.cc:237] T 51dffdf5066e42cd837177ce68e3cfdc P 7c7015c5b9374298b2d1035a7368b01b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad41dbaf35834010b9b4e6ab77ed009f" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 46061 } } peers { permanent_uuid: "7c7015c5b9374298b2d1035a7368b01b" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33675 } } peers { permanent_uuid: "c730379113054956917f1acf4e139a0b" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 35555 } }
I20260811 18:57:43.703195 5932 catalog_manager.cc:5495] T 51dffdf5066e42cd837177ce68e3cfdc P 7c7015c5b9374298b2d1035a7368b01b reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c7015c5b9374298b2d1035a7368b01b (127.29.112.130). New cstate: current_term: 1 leader_uuid: "7c7015c5b9374298b2d1035a7368b01b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad41dbaf35834010b9b4e6ab77ed009f" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 46061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c7015c5b9374298b2d1035a7368b01b" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33675 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c730379113054956917f1acf4e139a0b" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 35555 } health_report { overall_health: UNKNOWN } } }
I20260811 18:57:43.721964 6752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:43.719760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35438 (local address 127.29.112.130:33675)
0811 18:57:43.719951 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:43.719956 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:43.719973 (+ 17us) server_negotiation.cc:407] Connection header received
0811 18:57:43.720062 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:43.720065 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:43.720144 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:43.720232 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:43.721245 (+ 1013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:43.721251 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:43.721254 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:57:43.721275 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:43.721309 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:57:43.721322 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:43.721435 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:43.721592 (+ 157us) server_negotiation.cc:299] Negotiation successful
0811 18:57:43.721643 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 18:57:43.731024 6200 raft_consensus.cc:1261] T 51dffdf5066e42cd837177ce68e3cfdc P ad41dbaf35834010b9b4e6ab77ed009f [term 1 FOLLOWER]: Refusing update from remote peer 7c7015c5b9374298b2d1035a7368b01b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:57:43.731022 6653 raft_consensus.cc:1261] T 51dffdf5066e42cd837177ce68e3cfdc P c730379113054956917f1acf4e139a0b [term 1 FOLLOWER]: Refusing update from remote peer 7c7015c5b9374298b2d1035a7368b01b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:57:43.731949 6777 consensus_queue.cc:1035] T 51dffdf5066e42cd837177ce68e3cfdc P 7c7015c5b9374298b2d1035a7368b01b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c730379113054956917f1acf4e139a0b" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 35555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:43.732331 6768 consensus_queue.cc:1035] T 51dffdf5066e42cd837177ce68e3cfdc P 7c7015c5b9374298b2d1035a7368b01b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad41dbaf35834010b9b4e6ab77ed009f" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 46061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:43.744275 6782 mvcc.cc:205] Tried to move back new op lower bound from 7317400222632194048 to 7317400222511943680. Current Snapshot: MvccSnapshot[applied={T|T < 7317400222632194048}]
W20260811 18:57:43.744802 6730 tablet.cc:2369] T 51dffdf5066e42cd837177ce68e3cfdc P c730379113054956917f1acf4e139a0b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:57:43.761338 5930 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:59918:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:57:43.761694 5930 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:59918:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:57:43.765558 5930 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c20e0a7287794287a8d7243bb617c5f5: Sending DeleteTablet for 3 replicas of tablet 51dffdf5066e42cd837177ce68e3cfdc
I20260811 18:57:43.766726 6160 tablet_service.cc:1514] Processing DeleteTablet for tablet 51dffdf5066e42cd837177ce68e3cfdc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:43 UTC) from {username='slave'} at 127.0.0.1:50030
I20260811 18:57:43.767112 6619 tablet_service.cc:1514] Processing DeleteTablet for tablet 51dffdf5066e42cd837177ce68e3cfdc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:43 UTC) from {username='slave'} at 127.0.0.1:42222
I20260811 18:57:43.767293 6388 tablet_service.cc:1514] Processing DeleteTablet for tablet 51dffdf5066e42cd837177ce68e3cfdc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:43 UTC) from {username='slave'} at 127.0.0.1:35428
I20260811 18:57:43.769661 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 6050
I20260811 18:57:43.778874 6809 tablet_replica.cc:330] T 51dffdf5066e42cd837177ce68e3cfdc P c730379113054956917f1acf4e139a0b: stopping tablet replica
I20260811 18:57:43.779711 6809 raft_consensus.cc:2229] T 51dffdf5066e42cd837177ce68e3cfdc P c730379113054956917f1acf4e139a0b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:43.780505 6809 raft_consensus.cc:2258] T 51dffdf5066e42cd837177ce68e3cfdc P c730379113054956917f1acf4e139a0b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:43.783346 6809 ts_tablet_manager.cc:1905] T 51dffdf5066e42cd837177ce68e3cfdc P c730379113054956917f1acf4e139a0b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:43.793581 6809 ts_tablet_manager.cc:1918] T 51dffdf5066e42cd837177ce68e3cfdc P c730379113054956917f1acf4e139a0b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:57:43.793927 6809 log.cc:1196] T 51dffdf5066e42cd837177ce68e3cfdc P c730379113054956917f1acf4e139a0b: Deleting WAL directory at /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474615205622-30146-0/minicluster-data/ts-2/wal/wals/51dffdf5066e42cd837177ce68e3cfdc
I20260811 18:57:43.794751 6809 ts_tablet_manager.cc:1939] T 51dffdf5066e42cd837177ce68e3cfdc P c730379113054956917f1acf4e139a0b: Deleting consensus metadata
I20260811 18:57:43.797456 5908 catalog_manager.cc:4841] TS c730379113054956917f1acf4e139a0b (127.29.112.131:35555): tablet 51dffdf5066e42cd837177ce68e3cfdc (table test-table [id=944a26431e93496fab159127d377d26e]) successfully deleted
I20260811 18:57:43.798928 6810 tablet_replica.cc:330] T 51dffdf5066e42cd837177ce68e3cfdc P 7c7015c5b9374298b2d1035a7368b01b: stopping tablet replica
I20260811 18:57:43.799458 6810 raft_consensus.cc:2229] T 51dffdf5066e42cd837177ce68e3cfdc P 7c7015c5b9374298b2d1035a7368b01b [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:43.800242 6810 raft_consensus.cc:2258] T 51dffdf5066e42cd837177ce68e3cfdc P 7c7015c5b9374298b2d1035a7368b01b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:43.801465 6749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:43.771215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50034 (local address 127.29.112.129:46061)
0811 18:57:43.771396 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:43.771402 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:43.797165 (+ 25763us) server_negotiation.cc:407] Connection header received
0811 18:57:43.798347 (+ 1182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:43.798356 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:43.798522 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:43.798703 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:43.800282 (+ 1579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:43.800291 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:43.800299 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:57:43.800327 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:43.800379 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:57:43.800403 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:43.800549 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:43.800925 (+ 376us) server_negotiation.cc:299] Negotiation successful
0811 18:57:43.801015 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 18:57:43.802918 6263 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:57:43.805823 6810 ts_tablet_manager.cc:1905] T 51dffdf5066e42cd837177ce68e3cfdc P 7c7015c5b9374298b2d1035a7368b01b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:43.813431 6810 ts_tablet_manager.cc:1918] T 51dffdf5066e42cd837177ce68e3cfdc P 7c7015c5b9374298b2d1035a7368b01b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:57:43.813661 6810 log.cc:1196] T 51dffdf5066e42cd837177ce68e3cfdc P 7c7015c5b9374298b2d1035a7368b01b: Deleting WAL directory at /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474615205622-30146-0/minicluster-data/ts-1/wal/wals/51dffdf5066e42cd837177ce68e3cfdc
I20260811 18:57:43.814260 6810 ts_tablet_manager.cc:1939] T 51dffdf5066e42cd837177ce68e3cfdc P 7c7015c5b9374298b2d1035a7368b01b: Deleting consensus metadata
I20260811 18:57:43.816293 5908 catalog_manager.cc:4841] TS 7c7015c5b9374298b2d1035a7368b01b (127.29.112.130:33675): tablet 51dffdf5066e42cd837177ce68e3cfdc (table test-table [id=944a26431e93496fab159127d377d26e]) successfully deleted
I20260811 18:57:43.898591 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 6050
W20260811 18:57:43.928380 5901 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.112.129:46061 (error 108)
I20260811 18:57:43.929941 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 6279
I20260811 18:57:43.930437 6754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:43.929934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:46061 (local address 127.0.0.1:50042)
0811 18:57:43.930114 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:43.930257 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.129:46061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260811 18:57:43.930974 5901 catalog_manager.cc:4584] TS ad41dbaf35834010b9b4e6ab77ed009f (127.29.112.129:46061): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 51dffdf5066e42cd837177ce68e3cfdc: Network error: Client connection negotiation failed: client connection to 127.29.112.129:46061: connect: Connection refused (error 111)
I20260811 18:57:43.933066 6752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:43.930771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35454 (local address 127.29.112.130:33675)
0811 18:57:43.930972 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:43.930980 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:43.931002 (+ 22us) server_negotiation.cc:407] Connection header received
0811 18:57:43.931221 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:43.931228 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:43.931297 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:43.931417 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:43.932313 (+ 896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:43.932319 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:43.932324 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:57:43.932342 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:43.932379 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:57:43.932394 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:43.932525 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:43.932680 (+ 155us) server_negotiation.cc:299] Negotiation successful
0811 18:57:43.932743 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 18:57:43.933832 6500 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:57:43.984576 6754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:43.984078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:46061 (local address 127.0.0.1:50048)
0811 18:57:43.984304 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:43.984394 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.129:46061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260811 18:57:43.985764 6754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:43.985396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:46061 (local address 127.0.0.1:50064)
0811 18:57:43.985565 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:43.985649 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.129:46061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260811 18:57:44.046711 6754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:44.046136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:46061 (local address 127.0.0.1:50080)
0811 18:57:44.046368 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:44.046488 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.129:46061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260811 18:57:44.047914 6754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:44.047520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:46061 (local address 127.0.0.1:50096)
0811 18:57:44.047681 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:44.047771 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.129:46061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260811 18:57:44.052135 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 6279
I20260811 18:57:44.088341 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 6512
I20260811 18:57:44.091528 6755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:44.089191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42238 (local address 127.29.112.131:35555)
0811 18:57:44.089407 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:44.089413 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:44.089432 (+ 19us) server_negotiation.cc:407] Connection header received
0811 18:57:44.089560 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:44.089564 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:44.089637 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:44.089723 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:44.090757 (+ 1034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:44.090761 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:44.090765 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:57:44.090781 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:44.090813 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:57:44.090826 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:44.091040 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:44.091103 (+ 63us) server_negotiation.cc:299] Negotiation successful
0811 18:57:44.091183 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260811 18:57:44.092559 6718 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:57:44.128885 6754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:44.128379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:46061 (local address 127.0.0.1:50108)
0811 18:57:44.128622 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:44.128726 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.129:46061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260811 18:57:44.130016 6754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:44.129625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:46061 (local address 127.0.0.1:50112)
0811 18:57:44.129837 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:44.129909 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.129:46061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
I20260811 18:57:44.183110 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 6512
I20260811 18:57:44.207752 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 5874
I20260811 18:57:44.209003 5991 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:57:44.329427 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 5874
2026-08-11T18:57:44Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2932 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260811 18:57:44.381877 30146 mini_oidc.cc:152] Starting JWKS server
I20260811 18:57:44.384728 30146 webserver.cc:460] Webserver started at https://127.0.0.1:41733/ using document root <none> and password file <none>
I20260811 18:57:44.385090 30146 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 18:57:44.385773 30146 webserver.cc:460] Webserver started at http://127.0.0.1:37879/ using document root <none> and password file <none>
2026-08-11T18:57:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:57:44Z Disabled control of system clock
I20260811 18:57:44.420171 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474615205622-30146-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474615205622-30146-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474615205622-30146-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.190:35119
--webserver_interface=127.29.112.190
--webserver_port=0
--builtin_ntp_servers=127.29.112.148:40623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.190:35119
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41733/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474615205622-30146-0/testchainca.cert with env {}
W20260811 18:57:44.750666 6831 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:57:44.751019 6831 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:57:44.751189 6831 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:57:44.758543 6831 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:57:44.758689 6831 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:57:44.758780 6831 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:57:44.758886 6831 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:57:44.758970 6831 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 18:57:44.759073 6831 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 18:57:44.759156 6831 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:41733/jwks.json
I20260811 18:57:44.768986 6831 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:40623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474615205622-30146-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474615205622-30146-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.112.190:35119
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.190:35119
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41733/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.112.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474615205622-30146-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474615205622-30146-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:57:44.770339 6831 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:57:44.771653 6831 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:57:44.783834 6840 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:44.783957 6844 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:44.783922 6846 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:57:44.786142 6831 server_base.cc:1034] running on GCE node
I20260811 18:57:44.787284 6831 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:57:44.789309 6831 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:57:44.790629 6831 hybrid_clock.cc:648] HybridClock initialized: now 1786474664790567 us; error 47 us; skew 500 ppm
I20260811 18:57:44.791117 6831 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 18:57:44.793628 6831 webserver.cc:460] Webserver started at http://127.29.112.190:35537/ using document root <none> and password file <none>
I20260811 18:57:44.794342 6831 fs_manager.cc:359] Metadata directory not provided
I20260811 18:57:44.794468 6831 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:57:44.794800 6831 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:57:44.797726 6831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474615205622-30146-0/minicluster-data/master-0/data/instance:
uuid: "8e0f259f80ba4861bf7a32a37ff3118c"
format_stamp: "Formatted at 2026-08-11 18:57:44 on dist-test-slave-h6n0"
server_key: "a3e837a69b40e48451cd852ccce19fae"
server_key_iv: "89a0b650564ac65a124a61e1e888358a"
server_key_version: "encryptionkey@0"
I20260811 18:57:44.798615 6831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474615205622-30146-0/minicluster-data/master-0/wal/instance:
uuid: "8e0f259f80ba4861bf7a32a37ff3118c"
format_stamp: "Formatted at 2026-08-11 18:57:44 on dist-test-slave-h6n0"
server_key: "a3e837a69b40e48451cd852ccce19fae"
server_key_iv: "89a0b650564ac65a124a61e1e888358a"
server_key_version: "encryptionkey@0"
I20260811 18:57:44.804016 6831 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 18:57:44.808347 6855 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:44.810055 6831 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 18:57:44.810437 6831 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474615205622-30146-0/minicluster-data/master-0/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474615205622-30146-0/minicluster-data/master-0/wal
uuid: "8e0f259f80ba4861bf7a32a37ff3118c"
format_stamp: "Formatted at 2026-08-11 18:57:44 on dist-test-slave-h6n0"
server_key: "a3e837a69b40e48451cd852ccce19fae"
server_key_iv: "89a0b650564ac65a124a61e1e888358a"
server_key_version: "encryptionkey@0"
I20260811 18:57:44.810851 6831 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474615205622-30146-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474615205622-30146-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474615205622-30146-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:57:44.829813 6831 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:57:44.830782 6831 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:57:44.831153 6831 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:57:44.851747 6831 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.190:35119
I20260811 18:57:44.851758 6944 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.190:35119 every 8 connection(s)
I20260811 18:57:44.853207 6831 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
I20260811 18:57:44.856873 6945 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:44.861420 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 6831
I20260811 18:57:44.861642 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474615205622-30146-0/minicluster-data/master-0/wal/instance
I20260811 18:57:44.862466 30146 external_mini_cluster.cc:1428] Setting key 89c21d8cb16aceae7be7af06e6cbb584
I20260811 18:57:44.868816 6945 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e0f259f80ba4861bf7a32a37ff3118c: Bootstrap starting.
I20260811 18:57:44.872743 6945 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8e0f259f80ba4861bf7a32a37ff3118c: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:44.874151 6945 log.cc:824] T 00000000000000000000000000000000 P 8e0f259f80ba4861bf7a32a37ff3118c: Log is configured to *not* fsync() on all Append() calls
I20260811 18:57:44.880026 6945 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e0f259f80ba4861bf7a32a37ff3118c: No bootstrap required, opened a new log
I20260811 18:57:44.880416 6950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:44.864445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51064 (local address 127.29.112.190:35119)
0811 18:57:44.865485 (+ 1040us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:44.865496 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:44.865543 (+ 47us) server_negotiation.cc:407] Connection header received
0811 18:57:44.867413 (+ 1870us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:44.867451 (+ 38us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:44.867874 (+ 423us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:44.868214 (+ 340us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:44.868840 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:44.870232 (+ 1392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:44.872451 (+ 2219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:44.873234 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:44.873380 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:44.877808 (+ 4428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:44.877824 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:44.877843 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:57:44.878520 (+ 677us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:44.878659 (+ 139us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:57:44.878681 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:44.878803 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:44.879040 (+ 237us) server_negotiation.cc:299] Negotiation successful
0811 18:57:44.879430 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":631,"thread_start_us":398,"threads_started":1}
I20260811 18:57:44.884016 30146 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:57:44.888185 6945 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8e0f259f80ba4861bf7a32a37ff3118c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e0f259f80ba4861bf7a32a37ff3118c" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 35119 } }
I20260811 18:57:44.888540 6945 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8e0f259f80ba4861bf7a32a37ff3118c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:44.888656 6945 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8e0f259f80ba4861bf7a32a37ff3118c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e0f259f80ba4861bf7a32a37ff3118c, State: Initialized, Role: FOLLOWER
I20260811 18:57:44.889283 6945 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8e0f259f80ba4861bf7a32a37ff3118c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e0f259f80ba4861bf7a32a37ff3118c" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 35119 } }
I20260811 18:57:44.889562 6945 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8e0f259f80ba4861bf7a32a37ff3118c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:57:44.889711 6945 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8e0f259f80ba4861bf7a32a37ff3118c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:57:44.889878 6945 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8e0f259f80ba4861bf7a32a37ff3118c [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:44.892242 6945 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8e0f259f80ba4861bf7a32a37ff3118c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e0f259f80ba4861bf7a32a37ff3118c" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 35119 } }
I20260811 18:57:44.892766 6945 leader_election.cc:302] T 00000000000000000000000000000000 P 8e0f259f80ba4861bf7a32a37ff3118c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8e0f259f80ba4861bf7a32a37ff3118c; no voters:
I20260811 18:57:44.893481 6945 leader_election.cc:288] T 00000000000000000000000000000000 P 8e0f259f80ba4861bf7a32a37ff3118c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:57:44.893636 6952 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8e0f259f80ba4861bf7a32a37ff3118c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:44.894610 6952 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8e0f259f80ba4861bf7a32a37ff3118c [term 1 LEADER]: Becoming Leader. State: Replica: 8e0f259f80ba4861bf7a32a37ff3118c, State: Running, Role: LEADER
I20260811 18:57:44.895175 6952 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8e0f259f80ba4861bf7a32a37ff3118c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e0f259f80ba4861bf7a32a37ff3118c" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 35119 } }
I20260811 18:57:44.896636 6945 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8e0f259f80ba4861bf7a32a37ff3118c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:57:44.897929 6954 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e0f259f80ba4861bf7a32a37ff3118c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8e0f259f80ba4861bf7a32a37ff3118c. Latest consensus state: current_term: 1 leader_uuid: "8e0f259f80ba4861bf7a32a37ff3118c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e0f259f80ba4861bf7a32a37ff3118c" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 35119 } } }
I20260811 18:57:44.897965 6953 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e0f259f80ba4861bf7a32a37ff3118c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8e0f259f80ba4861bf7a32a37ff3118c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e0f259f80ba4861bf7a32a37ff3118c" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 35119 } } }
I20260811 18:57:44.898425 6954 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e0f259f80ba4861bf7a32a37ff3118c [sys.catalog]: This master's current role is: LEADER
I20260811 18:57:44.898425 6953 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e0f259f80ba4861bf7a32a37ff3118c [sys.catalog]: This master's current role is: LEADER
I20260811 18:57:44.903537 6959 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:57:44.909845 6959 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:57:44.920727 6959 catalog_manager.cc:1347] Generated new cluster ID: b89756c7ba3649078bb2b913993b4446
I20260811 18:57:44.920912 6959 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:57:44.939350 6959 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:57:44.940963 6959 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:57:44.949947 6959 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8e0f259f80ba4861bf7a32a37ff3118c: Generated new TSK 0
I20260811 18:57:44.950881 6959 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260811 18:57:45.000834 6994 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260811 18:57:45.001084 6994 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 18:57:45.001849 6950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:44.972063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51074 (local address 127.29.112.190:35119)
0811 18:57:44.972313 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:44.972321 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:44.972730 (+ 409us) server_negotiation.cc:407] Connection header received
0811 18:57:44.973153 (+ 423us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:44.973158 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:44.973228 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:44.973379 (+ 151us) server_negotiation.cc:226] Negotiated authn=JWT
0811 18:57:44.973995 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:44.974815 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:44.976936 (+ 2121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:44.977746 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:44.977875 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:44.978511 (+ 636us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 18:57:45.001312 (+ 22801us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41733/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0811 18:57:45.001625 (+ 313us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:51074: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41733/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":69}
W20260811 18:57:45.001993 6950 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:51074: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41733/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260811 18:57:45.003947 6993 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:57:44.971944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:35119 (local address 127.0.0.1:51074)
0811 18:57:44.972622 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:44.972644 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:44.972826 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:44.973401 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:44.973407 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:44.973420 (+ 13us) client_negotiation.cc:189] Negotiated authn=JWT
0811 18:57:44.973836 (+ 416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:44.973847 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:44.974955 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:44.974960 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:44.976772 (+ 1812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:44.976786 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:44.977967 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:44.977974 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:44.978169 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:44.978243 (+ 74us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 18:57:45.001624 (+ 23381us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41733/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0811 18:57:45.002005 (+ 381us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.29.112.190:35119: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41733/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":325,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.29.112.190:35119: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41733/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260811 18:57:45.006122 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 6831
I20260811 18:57:45.007345 6934 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:57:45.107542 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 6831
2026-08-11T18:57:45Z chronyd exiting
I20260811 18:57:45.168756 30146 test_util.cc:175] -----------------------------------------------
I20260811 18:57:45.168931 30146 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474615205622-30146-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (791 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260811 18:57:45.173127 30146 mini_oidc.cc:152] Starting JWKS server
I20260811 18:57:45.175150 30146 webserver.cc:460] Webserver started at https://127.0.0.1:35407/ using document root <none> and password file <none>
I20260811 18:57:45.175426 30146 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 18:57:45.176092 30146 webserver.cc:460] Webserver started at http://127.0.0.1:32797/ using document root <none> and password file <none>
2026-08-11T18:57:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:57:45Z Disabled control of system clock
I20260811 18:57:45.210038 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786474615205622-30146-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786474615205622-30146-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786474615205622-30146-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.190:41247
--webserver_interface=127.29.112.190
--webserver_port=0
--builtin_ntp_servers=127.29.112.148:45483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.190:41247
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35407/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786474615205622-30146-0/testchainca.cert with env {}
W20260811 18:57:45.555451 7005 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:57:45.555816 7005 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:57:45.555980 7005 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:57:45.564883 7005 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:57:45.565148 7005 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:57:45.565258 7005 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:57:45.565388 7005 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:57:45.565495 7005 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 18:57:45.565614 7005 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 18:57:45.565706 7005 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:35407/jwks.json
I20260811 18:57:45.579221 7005 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:45483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786474615205622-30146-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786474615205622-30146-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.112.190:41247
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.190:41247
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35407/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.112.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786474615205622-30146-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786474615205622-30146-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:57:45.580596 7005 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:57:45.581899 7005 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:57:45.594628 7015 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:45.594614 7016 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:45.594508 7019 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:57:45.595332 7005 server_base.cc:1034] running on GCE node
I20260811 18:57:45.596189 7005 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:57:45.597635 7005 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:57:45.598887 7005 hybrid_clock.cc:648] HybridClock initialized: now 1786474665598834 us; error 39 us; skew 500 ppm
I20260811 18:57:45.599288 7005 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 18:57:45.601796 7005 webserver.cc:460] Webserver started at http://127.29.112.190:43087/ using document root <none> and password file <none>
I20260811 18:57:45.602506 7005 fs_manager.cc:359] Metadata directory not provided
I20260811 18:57:45.602698 7005 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:57:45.603049 7005 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:57:45.605945 7005 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786474615205622-30146-0/minicluster-data/master-0/data/instance:
uuid: "88c06702506b49e3b8b23d013f617020"
format_stamp: "Formatted at 2026-08-11 18:57:45 on dist-test-slave-h6n0"
server_key: "c0f78eb32f590f6af49da244541ef326"
server_key_iv: "c3c85f5009c2820e2e88287c12cb1de4"
server_key_version: "encryptionkey@0"
I20260811 18:57:45.606865 7005 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786474615205622-30146-0/minicluster-data/master-0/wal/instance:
uuid: "88c06702506b49e3b8b23d013f617020"
format_stamp: "Formatted at 2026-08-11 18:57:45 on dist-test-slave-h6n0"
server_key: "c0f78eb32f590f6af49da244541ef326"
server_key_iv: "c3c85f5009c2820e2e88287c12cb1de4"
server_key_version: "encryptionkey@0"
I20260811 18:57:45.612939 7005 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260811 18:57:45.617220 7026 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:45.619053 7005 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260811 18:57:45.619289 7005 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786474615205622-30146-0/minicluster-data/master-0/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786474615205622-30146-0/minicluster-data/master-0/wal
uuid: "88c06702506b49e3b8b23d013f617020"
format_stamp: "Formatted at 2026-08-11 18:57:45 on dist-test-slave-h6n0"
server_key: "c0f78eb32f590f6af49da244541ef326"
server_key_iv: "c3c85f5009c2820e2e88287c12cb1de4"
server_key_version: "encryptionkey@0"
I20260811 18:57:45.619553 7005 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786474615205622-30146-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786474615205622-30146-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786474615205622-30146-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:57:45.633270 7005 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:57:45.634163 7005 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:57:45.634510 7005 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:57:45.657871 7005 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.190:41247
I20260811 18:57:45.657900 7112 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.190:41247 every 8 connection(s)
I20260811 18:57:45.659711 7005 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
I20260811 18:57:45.663537 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 7005
I20260811 18:57:45.663784 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786474615205622-30146-0/minicluster-data/master-0/wal/instance
I20260811 18:57:45.663807 7114 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:45.664469 30146 external_mini_cluster.cc:1428] Setting key eadda49905732540deb7886e7e34d90c
I20260811 18:57:45.678565 7114 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88c06702506b49e3b8b23d013f617020: Bootstrap starting.
I20260811 18:57:45.681449 7119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:45.666492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34610 (local address 127.29.112.190:41247)
0811 18:57:45.667695 (+ 1203us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:45.667711 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:45.667764 (+ 53us) server_negotiation.cc:407] Connection header received
0811 18:57:45.670591 (+ 2827us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:45.670620 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:45.671246 (+ 626us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:45.671706 (+ 460us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:45.672500 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:45.674608 (+ 2108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:45.676161 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:45.677191 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:45.677397 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:45.678607 (+ 1210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:45.678619 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:45.678636 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:57:45.679338 (+ 702us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:45.679441 (+ 103us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:57:45.679469 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:45.679588 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:45.679813 (+ 225us) server_negotiation.cc:299] Negotiation successful
0811 18:57:45.680132 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":635,"thread_start_us":369,"threads_started":1}
I20260811 18:57:45.682715 7114 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 88c06702506b49e3b8b23d013f617020: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:45.683862 7114 log.cc:824] T 00000000000000000000000000000000 P 88c06702506b49e3b8b23d013f617020: Log is configured to *not* fsync() on all Append() calls
I20260811 18:57:45.685664 30146 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:57:45.686995 7114 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88c06702506b49e3b8b23d013f617020: No bootstrap required, opened a new log
I20260811 18:57:45.694957 7114 raft_consensus.cc:348] T 00000000000000000000000000000000 P 88c06702506b49e3b8b23d013f617020 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88c06702506b49e3b8b23d013f617020" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 41247 } }
I20260811 18:57:45.695334 7114 raft_consensus.cc:374] T 00000000000000000000000000000000 P 88c06702506b49e3b8b23d013f617020 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:45.695446 7114 raft_consensus.cc:729] T 00000000000000000000000000000000 P 88c06702506b49e3b8b23d013f617020 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88c06702506b49e3b8b23d013f617020, State: Initialized, Role: FOLLOWER
I20260811 18:57:45.696079 7114 consensus_queue.cc:260] T 00000000000000000000000000000000 P 88c06702506b49e3b8b23d013f617020 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88c06702506b49e3b8b23d013f617020" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 41247 } }
I20260811 18:57:45.696378 7114 raft_consensus.cc:388] T 00000000000000000000000000000000 P 88c06702506b49e3b8b23d013f617020 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:57:45.696547 7114 raft_consensus.cc:482] T 00000000000000000000000000000000 P 88c06702506b49e3b8b23d013f617020 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:57:45.696722 7114 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 88c06702506b49e3b8b23d013f617020 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:45.698989 7114 raft_consensus.cc:504] T 00000000000000000000000000000000 P 88c06702506b49e3b8b23d013f617020 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88c06702506b49e3b8b23d013f617020" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 41247 } }
I20260811 18:57:45.699489 7114 leader_election.cc:302] T 00000000000000000000000000000000 P 88c06702506b49e3b8b23d013f617020 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 88c06702506b49e3b8b23d013f617020; no voters:
I20260811 18:57:45.700134 7114 leader_election.cc:288] T 00000000000000000000000000000000 P 88c06702506b49e3b8b23d013f617020 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:57:45.700335 7130 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 88c06702506b49e3b8b23d013f617020 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:45.701241 7130 raft_consensus.cc:686] T 00000000000000000000000000000000 P 88c06702506b49e3b8b23d013f617020 [term 1 LEADER]: Becoming Leader. State: Replica: 88c06702506b49e3b8b23d013f617020, State: Running, Role: LEADER
I20260811 18:57:45.701846 7130 consensus_queue.cc:237] T 00000000000000000000000000000000 P 88c06702506b49e3b8b23d013f617020 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88c06702506b49e3b8b23d013f617020" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 41247 } }
I20260811 18:57:45.702343 7114 sys_catalog.cc:564] T 00000000000000000000000000000000 P 88c06702506b49e3b8b23d013f617020 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:57:45.704707 7133 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88c06702506b49e3b8b23d013f617020 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 88c06702506b49e3b8b23d013f617020. Latest consensus state: current_term: 1 leader_uuid: "88c06702506b49e3b8b23d013f617020" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88c06702506b49e3b8b23d013f617020" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 41247 } } }
I20260811 18:57:45.705148 7133 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88c06702506b49e3b8b23d013f617020 [sys.catalog]: This master's current role is: LEADER
I20260811 18:57:45.705641 7132 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88c06702506b49e3b8b23d013f617020 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "88c06702506b49e3b8b23d013f617020" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88c06702506b49e3b8b23d013f617020" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 41247 } } }
I20260811 18:57:45.706079 7132 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88c06702506b49e3b8b23d013f617020 [sys.catalog]: This master's current role is: LEADER
I20260811 18:57:45.707842 7142 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:57:45.713333 7142 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:57:45.723978 7142 catalog_manager.cc:1347] Generated new cluster ID: 6885d4c198414e4781224d52bda3d969
I20260811 18:57:45.724123 7142 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:57:45.739019 7142 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:57:45.740759 7142 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:57:45.764215 7142 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 88c06702506b49e3b8b23d013f617020: Generated new TSK 0
I20260811 18:57:45.765341 7142 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260811 18:57:45.810017 7171 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260811 18:57:45.810169 7171 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 18:57:45.811633 7119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:45.790127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34624 (local address 127.29.112.190:41247)
0811 18:57:45.790330 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:45.790338 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:45.790921 (+ 583us) server_negotiation.cc:407] Connection header received
0811 18:57:45.791145 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:45.791148 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:45.791221 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:45.791339 (+ 118us) server_negotiation.cc:226] Negotiated authn=JWT
0811 18:57:45.791884 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:45.792654 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:45.794724 (+ 2070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:45.795247 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:45.795361 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:45.795995 (+ 634us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 18:57:45.810847 (+ 14852us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35407/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0811 18:57:45.811386 (+ 539us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:34624: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35407/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":66}
W20260811 18:57:45.811864 7119 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:34624: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35407/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260811 18:57:45.811916 7169 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:57:45.790020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:41247 (local address 127.0.0.1:34624)
0811 18:57:45.790811 (+ 791us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:45.790831 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:45.790987 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:45.791379 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:45.791384 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:45.791397 (+ 13us) client_negotiation.cc:189] Negotiated authn=JWT
0811 18:57:45.791702 (+ 305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:45.791714 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:45.792793 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:45.792799 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:45.794550 (+ 1751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:45.794568 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:45.795424 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:45.795431 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:45.795632 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:45.795681 (+ 49us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 18:57:45.811212 (+ 15531us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35407/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0811 18:57:45.811512 (+ 300us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.29.112.190:41247: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35407/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":644,"thread_start_us":502,"threads_started":1}
I20260811 18:57:45.813340 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 7005
I20260811 18:57:45.814635 7105 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:57:45.910831 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 7005
2026-08-11T18:57:45Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (801 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260811 18:57:45.974661 30146 mini_oidc.cc:152] Starting JWKS server
I20260811 18:57:45.976691 30146 webserver.cc:460] Webserver started at https://127.0.0.1:44469/ using document root <none> and password file <none>
I20260811 18:57:45.976902 30146 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 18:57:45.977515 30146 webserver.cc:460] Webserver started at http://127.0.0.1:38931/ using document root <none> and password file <none>
2026-08-11T18:57:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:57:45Z Disabled control of system clock
I20260811 18:57:46.013469 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786474615205622-30146-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786474615205622-30146-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786474615205622-30146-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.190:45765
--webserver_interface=127.29.112.190
--webserver_port=0
--builtin_ntp_servers=127.29.112.148:34131
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.190:45765
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44469/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260811 18:57:46.355172 7181 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:57:46.355501 7181 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:57:46.355691 7181 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:57:46.362984 7181 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:57:46.363094 7181 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:57:46.363185 7181 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:57:46.363282 7181 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:57:46.363366 7181 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 18:57:46.363478 7181 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 18:57:46.363550 7181 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:44469/jwks.json
I20260811 18:57:46.372639 7181 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:34131
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786474615205622-30146-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786474615205622-30146-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.112.190:45765
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.190:45765
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44469/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.112.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786474615205622-30146-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:57:46.373911 7181 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:57:46.375696 7181 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:57:46.387411 7191 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:46.387497 7194 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:46.387456 7199 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:57:46.389117 7181 server_base.cc:1034] running on GCE node
I20260811 18:57:46.389889 7181 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:57:46.391287 7181 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:57:46.392509 7181 hybrid_clock.cc:648] HybridClock initialized: now 1786474666392486 us; error 56 us; skew 500 ppm
I20260811 18:57:46.392846 7181 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 18:57:46.395217 7181 webserver.cc:460] Webserver started at http://127.29.112.190:39843/ using document root <none> and password file <none>
I20260811 18:57:46.395931 7181 fs_manager.cc:359] Metadata directory not provided
I20260811 18:57:46.396061 7181 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:57:46.396405 7181 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:57:46.399348 7181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786474615205622-30146-0/minicluster-data/master-0/data/instance:
uuid: "85383f606b8f484b89165d24e1bb3205"
format_stamp: "Formatted at 2026-08-11 18:57:46 on dist-test-slave-h6n0"
server_key: "4594d1f42318808ca0af793937b54e28"
server_key_iv: "1a7ec0f9f936c3decdf45e730d7eb325"
server_key_version: "encryptionkey@0"
I20260811 18:57:46.400151 7181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786474615205622-30146-0/minicluster-data/master-0/wal/instance:
uuid: "85383f606b8f484b89165d24e1bb3205"
format_stamp: "Formatted at 2026-08-11 18:57:46 on dist-test-slave-h6n0"
server_key: "4594d1f42318808ca0af793937b54e28"
server_key_iv: "1a7ec0f9f936c3decdf45e730d7eb325"
server_key_version: "encryptionkey@0"
I20260811 18:57:46.405242 7181 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 18:57:46.409301 7205 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:46.411729 7181 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.001s
I20260811 18:57:46.412111 7181 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786474615205622-30146-0/minicluster-data/master-0/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786474615205622-30146-0/minicluster-data/master-0/wal
uuid: "85383f606b8f484b89165d24e1bb3205"
format_stamp: "Formatted at 2026-08-11 18:57:46 on dist-test-slave-h6n0"
server_key: "4594d1f42318808ca0af793937b54e28"
server_key_iv: "1a7ec0f9f936c3decdf45e730d7eb325"
server_key_version: "encryptionkey@0"
I20260811 18:57:46.412626 7181 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786474615205622-30146-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786474615205622-30146-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786474615205622-30146-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:57:46.425444 7181 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:57:46.426374 7181 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:57:46.426746 7181 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:57:46.448534 7181 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.190:45765
I20260811 18:57:46.448558 7302 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.190:45765 every 8 connection(s)
I20260811 18:57:46.449931 7181 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
I20260811 18:57:46.453740 7304 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:46.457659 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 7181
I20260811 18:57:46.457880 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786474615205622-30146-0/minicluster-data/master-0/wal/instance
I20260811 18:57:46.458751 30146 external_mini_cluster.cc:1428] Setting key 6fbefbde0932aaa68a8553131d9f6402
I20260811 18:57:46.466743 7304 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85383f606b8f484b89165d24e1bb3205: Bootstrap starting.
I20260811 18:57:46.470391 7304 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 85383f606b8f484b89165d24e1bb3205: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:46.471745 7304 log.cc:824] T 00000000000000000000000000000000 P 85383f606b8f484b89165d24e1bb3205: Log is configured to *not* fsync() on all Append() calls
I20260811 18:57:46.472370 7308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:46.460697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50002 (local address 127.29.112.190:45765)
0811 18:57:46.461671 (+ 974us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:46.461681 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:46.461719 (+ 38us) server_negotiation.cc:407] Connection header received
0811 18:57:46.463394 (+ 1675us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:46.463415 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:46.463842 (+ 427us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:46.464209 (+ 367us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:46.464798 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:46.466018 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:46.467383 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:46.468032 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:46.468291 (+ 259us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:46.469624 (+ 1333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:46.469631 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:46.469643 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:57:46.470169 (+ 526us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:46.470253 (+ 84us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:57:46.470274 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:46.470412 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:46.470666 (+ 254us) server_negotiation.cc:299] Negotiation successful
0811 18:57:46.471024 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":587,"thread_start_us":349,"threads_started":1}
I20260811 18:57:46.474856 7304 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85383f606b8f484b89165d24e1bb3205: No bootstrap required, opened a new log
I20260811 18:57:46.476208 30146 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:57:46.485267 7304 raft_consensus.cc:348] T 00000000000000000000000000000000 P 85383f606b8f484b89165d24e1bb3205 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85383f606b8f484b89165d24e1bb3205" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 45765 } }
I20260811 18:57:46.485782 7304 raft_consensus.cc:374] T 00000000000000000000000000000000 P 85383f606b8f484b89165d24e1bb3205 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:46.485929 7304 raft_consensus.cc:729] T 00000000000000000000000000000000 P 85383f606b8f484b89165d24e1bb3205 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85383f606b8f484b89165d24e1bb3205, State: Initialized, Role: FOLLOWER
I20260811 18:57:46.486677 7304 consensus_queue.cc:260] T 00000000000000000000000000000000 P 85383f606b8f484b89165d24e1bb3205 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85383f606b8f484b89165d24e1bb3205" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 45765 } }
I20260811 18:57:46.487039 7304 raft_consensus.cc:388] T 00000000000000000000000000000000 P 85383f606b8f484b89165d24e1bb3205 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:57:46.487238 7304 raft_consensus.cc:482] T 00000000000000000000000000000000 P 85383f606b8f484b89165d24e1bb3205 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:57:46.487416 7304 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 85383f606b8f484b89165d24e1bb3205 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:46.489681 7304 raft_consensus.cc:504] T 00000000000000000000000000000000 P 85383f606b8f484b89165d24e1bb3205 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85383f606b8f484b89165d24e1bb3205" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 45765 } }
I20260811 18:57:46.490586 7304 leader_election.cc:302] T 00000000000000000000000000000000 P 85383f606b8f484b89165d24e1bb3205 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 85383f606b8f484b89165d24e1bb3205; no voters:
I20260811 18:57:46.491410 7304 leader_election.cc:288] T 00000000000000000000000000000000 P 85383f606b8f484b89165d24e1bb3205 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:57:46.491477 7311 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 85383f606b8f484b89165d24e1bb3205 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:46.492635 7311 raft_consensus.cc:686] T 00000000000000000000000000000000 P 85383f606b8f484b89165d24e1bb3205 [term 1 LEADER]: Becoming Leader. State: Replica: 85383f606b8f484b89165d24e1bb3205, State: Running, Role: LEADER
I20260811 18:57:46.493232 7311 consensus_queue.cc:237] T 00000000000000000000000000000000 P 85383f606b8f484b89165d24e1bb3205 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85383f606b8f484b89165d24e1bb3205" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 45765 } }
I20260811 18:57:46.494021 7304 sys_catalog.cc:564] T 00000000000000000000000000000000 P 85383f606b8f484b89165d24e1bb3205 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:57:46.496356 7312 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85383f606b8f484b89165d24e1bb3205 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "85383f606b8f484b89165d24e1bb3205" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85383f606b8f484b89165d24e1bb3205" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 45765 } } }
I20260811 18:57:46.496335 7313 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85383f606b8f484b89165d24e1bb3205 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 85383f606b8f484b89165d24e1bb3205. Latest consensus state: current_term: 1 leader_uuid: "85383f606b8f484b89165d24e1bb3205" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85383f606b8f484b89165d24e1bb3205" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 45765 } } }
I20260811 18:57:46.496774 7312 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85383f606b8f484b89165d24e1bb3205 [sys.catalog]: This master's current role is: LEADER
I20260811 18:57:46.496845 7313 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85383f606b8f484b89165d24e1bb3205 [sys.catalog]: This master's current role is: LEADER
I20260811 18:57:46.503366 7324 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:57:46.509668 7324 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:57:46.521811 7324 catalog_manager.cc:1347] Generated new cluster ID: d178dc7062bd4e8488116f65bbcffff4
I20260811 18:57:46.521992 7324 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:57:46.543357 7324 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:57:46.544853 7324 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:57:46.558418 7324 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 85383f606b8f484b89165d24e1bb3205: Generated new TSK 0
I20260811 18:57:46.559530 7324 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:57:46.592254 7308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:46.582581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50006 (local address 127.29.112.190:45765)
0811 18:57:46.582797 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:46.582805 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:46.583289 (+ 484us) server_negotiation.cc:407] Connection header received
0811 18:57:46.583548 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:46.583554 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:46.583650 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:46.583833 (+ 183us) server_negotiation.cc:226] Negotiated authn=JWT
0811 18:57:46.584853 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:46.586040 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:46.587614 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:46.588353 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:46.588496 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:46.589028 (+ 532us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 18:57:46.591556 (+ 2528us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44469/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0811 18:57:46.591911 (+ 355us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:50006: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44469/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":71}
W20260811 18:57:46.592411 7308 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:50006: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44469/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260811 18:57:46.594904 7352 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:57:46.582445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:45765 (local address 127.0.0.1:50006)
0811 18:57:46.583180 (+ 735us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:46.583198 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:46.583357 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:46.583957 (+ 600us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:46.583963 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:46.583980 (+ 17us) client_negotiation.cc:189] Negotiated authn=JWT
0811 18:57:46.584623 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:46.584643 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:46.586247 (+ 1604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:46.586253 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:46.587464 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:46.587476 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:46.588580 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:46.588586 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:46.588719 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:46.588757 (+ 38us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 18:57:46.591825 (+ 3068us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44469/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0811 18:57:46.594201 (+ 2376us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.29.112.190:45765: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44469/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":594,"mutex_wait_us":1904,"thread_start_us":417,"threads_started":1}
I20260811 18:57:46.598848 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 7181
I20260811 18:57:46.600970 7296 generic_service.cc:196] Checking for leaks (request via RPC)
E20260811 18:57:46.606716 7353 webserver.cc:533] Webserver: sslize failed:
E20260811 18:57:46.606807 7353 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 18:57:46.697963 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 7181
2026-08-11T18:57:46Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (786 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-11T18:57:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:57:46Z Disabled control of system clock
I20260811 18:57:46.795745 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786474615205622-30146-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786474615205622-30146-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786474615205622-30146-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.190:33057
--webserver_interface=127.29.112.190
--webserver_port=0
--builtin_ntp_servers=127.29.112.148:34967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.190:33057
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786474615205622-30146-0/testchainca.cert with env {}
W20260811 18:57:47.139009 7362 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:57:47.139353 7362 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:57:47.139452 7362 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:57:47.147723 7362 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:57:47.147900 7362 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:57:47.147945 7362 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:57:47.148010 7362 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:57:47.148077 7362 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 18:57:47.148168 7362 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 18:57:47.148221 7362 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260811 18:57:47.159840 7362 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:34967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786474615205622-30146-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786474615205622-30146-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.112.190:33057
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.190:33057
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.112.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786474615205622-30146-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786474615205622-30146-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:57:47.161486 7362 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:57:47.163317 7362 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:57:47.175704 7372 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:47.175673 7374 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:47.175613 7376 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:57:47.176842 7362 server_base.cc:1034] running on GCE node
I20260811 18:57:47.177604 7362 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:57:47.179725 7362 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:57:47.180959 7362 hybrid_clock.cc:648] HybridClock initialized: now 1786474667180915 us; error 34 us; skew 500 ppm
I20260811 18:57:47.181391 7362 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 18:57:47.184289 7362 webserver.cc:460] Webserver started at http://127.29.112.190:41731/ using document root <none> and password file <none>
I20260811 18:57:47.185055 7362 fs_manager.cc:359] Metadata directory not provided
I20260811 18:57:47.185168 7362 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:57:47.185549 7362 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:57:47.189206 7362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786474615205622-30146-0/minicluster-data/master-0/data/instance:
uuid: "d161aab4760a44139d13dd1fbf9a4753"
format_stamp: "Formatted at 2026-08-11 18:57:47 on dist-test-slave-h6n0"
server_key: "c420402a19eef2c262551588f954dbf4"
server_key_iv: "a1f36e46ad7b90958a6c174d2f9f0fa2"
server_key_version: "encryptionkey@0"
I20260811 18:57:47.190124 7362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786474615205622-30146-0/minicluster-data/master-0/wal/instance:
uuid: "d161aab4760a44139d13dd1fbf9a4753"
format_stamp: "Formatted at 2026-08-11 18:57:47 on dist-test-slave-h6n0"
server_key: "c420402a19eef2c262551588f954dbf4"
server_key_iv: "a1f36e46ad7b90958a6c174d2f9f0fa2"
server_key_version: "encryptionkey@0"
I20260811 18:57:47.195969 7362 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260811 18:57:47.200263 7386 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:47.201776 7362 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.002s
I20260811 18:57:47.201994 7362 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786474615205622-30146-0/minicluster-data/master-0/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786474615205622-30146-0/minicluster-data/master-0/wal
uuid: "d161aab4760a44139d13dd1fbf9a4753"
format_stamp: "Formatted at 2026-08-11 18:57:47 on dist-test-slave-h6n0"
server_key: "c420402a19eef2c262551588f954dbf4"
server_key_iv: "a1f36e46ad7b90958a6c174d2f9f0fa2"
server_key_version: "encryptionkey@0"
I20260811 18:57:47.202235 7362 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786474615205622-30146-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786474615205622-30146-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786474615205622-30146-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:57:47.217947 7362 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:57:47.219347 7362 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:57:47.219732 7362 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:57:47.240559 7362 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.190:33057
I20260811 18:57:47.240584 7472 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.190:33057 every 8 connection(s)
I20260811 18:57:47.241948 7362 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
I20260811 18:57:47.245708 7473 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:47.248342 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 7362
I20260811 18:57:47.248588 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786474615205622-30146-0/minicluster-data/master-0/wal/instance
I20260811 18:57:47.249238 30146 external_mini_cluster.cc:1428] Setting key ee0a6a0033c4d8e8487f3fa2d37ef1de
I20260811 18:57:47.259088 7473 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d161aab4760a44139d13dd1fbf9a4753: Bootstrap starting.
I20260811 18:57:47.262857 7473 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d161aab4760a44139d13dd1fbf9a4753: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:47.264204 7473 log.cc:824] T 00000000000000000000000000000000 P d161aab4760a44139d13dd1fbf9a4753: Log is configured to *not* fsync() on all Append() calls
I20260811 18:57:47.264624 7478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:47.251263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58774 (local address 127.29.112.190:33057)
0811 18:57:47.252286 (+ 1023us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:47.252301 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:47.252357 (+ 56us) server_negotiation.cc:407] Connection header received
0811 18:57:47.254574 (+ 2217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:47.254608 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:47.255201 (+ 593us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:47.255665 (+ 464us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:47.256186 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:47.258160 (+ 1974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:47.259542 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:47.260502 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:47.260734 (+ 232us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:47.261812 (+ 1078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:47.261821 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:47.261838 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:57:47.262600 (+ 762us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:47.262673 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:57:47.262708 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:47.262835 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:47.263033 (+ 198us) server_negotiation.cc:299] Negotiation successful
0811 18:57:47.263355 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":554,"thread_start_us":329,"threads_started":1}
I20260811 18:57:47.267307 7473 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d161aab4760a44139d13dd1fbf9a4753: No bootstrap required, opened a new log
I20260811 18:57:47.268285 30146 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:57:47.274753 7473 raft_consensus.cc:348] T 00000000000000000000000000000000 P d161aab4760a44139d13dd1fbf9a4753 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d161aab4760a44139d13dd1fbf9a4753" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 33057 } }
I20260811 18:57:47.275125 7473 raft_consensus.cc:374] T 00000000000000000000000000000000 P d161aab4760a44139d13dd1fbf9a4753 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:47.275235 7473 raft_consensus.cc:729] T 00000000000000000000000000000000 P d161aab4760a44139d13dd1fbf9a4753 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d161aab4760a44139d13dd1fbf9a4753, State: Initialized, Role: FOLLOWER
I20260811 18:57:47.275811 7473 consensus_queue.cc:260] T 00000000000000000000000000000000 P d161aab4760a44139d13dd1fbf9a4753 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d161aab4760a44139d13dd1fbf9a4753" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 33057 } }
I20260811 18:57:47.276037 7473 raft_consensus.cc:388] T 00000000000000000000000000000000 P d161aab4760a44139d13dd1fbf9a4753 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:57:47.276180 7473 raft_consensus.cc:482] T 00000000000000000000000000000000 P d161aab4760a44139d13dd1fbf9a4753 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:57:47.276347 7473 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d161aab4760a44139d13dd1fbf9a4753 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:47.278465 7473 raft_consensus.cc:504] T 00000000000000000000000000000000 P d161aab4760a44139d13dd1fbf9a4753 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d161aab4760a44139d13dd1fbf9a4753" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 33057 } }
I20260811 18:57:47.278987 7473 leader_election.cc:302] T 00000000000000000000000000000000 P d161aab4760a44139d13dd1fbf9a4753 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d161aab4760a44139d13dd1fbf9a4753; no voters:
I20260811 18:57:47.279580 7473 leader_election.cc:288] T 00000000000000000000000000000000 P d161aab4760a44139d13dd1fbf9a4753 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:57:47.279744 7482 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d161aab4760a44139d13dd1fbf9a4753 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:47.280993 7482 raft_consensus.cc:686] T 00000000000000000000000000000000 P d161aab4760a44139d13dd1fbf9a4753 [term 1 LEADER]: Becoming Leader. State: Replica: d161aab4760a44139d13dd1fbf9a4753, State: Running, Role: LEADER
I20260811 18:57:47.281508 7482 consensus_queue.cc:237] T 00000000000000000000000000000000 P d161aab4760a44139d13dd1fbf9a4753 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d161aab4760a44139d13dd1fbf9a4753" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 33057 } }
I20260811 18:57:47.281754 7473 sys_catalog.cc:564] T 00000000000000000000000000000000 P d161aab4760a44139d13dd1fbf9a4753 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:57:47.284550 7485 sys_catalog.cc:455] T 00000000000000000000000000000000 P d161aab4760a44139d13dd1fbf9a4753 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d161aab4760a44139d13dd1fbf9a4753. Latest consensus state: current_term: 1 leader_uuid: "d161aab4760a44139d13dd1fbf9a4753" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d161aab4760a44139d13dd1fbf9a4753" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 33057 } } }
I20260811 18:57:47.284528 7483 sys_catalog.cc:455] T 00000000000000000000000000000000 P d161aab4760a44139d13dd1fbf9a4753 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d161aab4760a44139d13dd1fbf9a4753" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d161aab4760a44139d13dd1fbf9a4753" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 33057 } } }
I20260811 18:57:47.285028 7483 sys_catalog.cc:458] T 00000000000000000000000000000000 P d161aab4760a44139d13dd1fbf9a4753 [sys.catalog]: This master's current role is: LEADER
I20260811 18:57:47.285028 7485 sys_catalog.cc:458] T 00000000000000000000000000000000 P d161aab4760a44139d13dd1fbf9a4753 [sys.catalog]: This master's current role is: LEADER
I20260811 18:57:47.286780 7495 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:57:47.292846 7495 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:57:47.302266 7495 catalog_manager.cc:1347] Generated new cluster ID: ed8f527cc21e4d14ac805a5d1a91b112
I20260811 18:57:47.302415 7495 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:57:47.334213 7495 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:57:47.335597 7495 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:57:47.348623 7495 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d161aab4760a44139d13dd1fbf9a4753: Generated new TSK 0
I20260811 18:57:47.349839 7495 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:57:47.440335 7478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:47.372988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58776 (local address 127.29.112.190:33057)
0811 18:57:47.373336 (+ 348us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:47.373344 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:47.373536 (+ 192us) server_negotiation.cc:407] Connection header received
0811 18:57:47.373931 (+ 395us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:47.373940 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:47.374037 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:47.374233 (+ 196us) server_negotiation.cc:226] Negotiated authn=JWT
0811 18:57:47.374898 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:47.375738 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:47.377302 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:47.377803 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:47.377953 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:47.378568 (+ 615us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 18:57:47.439525 (+ 60957us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0811 18:57:47.439979 (+ 454us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:58776: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":136}
W20260811 18:57:47.440501 7478 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:58776: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260811 18:57:47.440552 7521 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:57:47.372457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:33057 (local address 127.0.0.1:58776)
0811 18:57:47.373386 (+ 929us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:47.373417 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:47.373648 (+ 231us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:47.374261 (+ 613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:47.374267 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:47.374279 (+ 12us) client_negotiation.cc:189] Negotiated authn=JWT
0811 18:57:47.374713 (+ 434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:47.374725 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:47.375900 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:47.375906 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:47.377133 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:47.377145 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:47.378004 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:47.378011 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:47.378174 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:47.378217 (+ 43us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 18:57:47.439880 (+ 61663us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0811 18:57:47.440157 (+ 277us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.29.112.190:33057: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":749,"thread_start_us":454,"threads_started":1}
I20260811 18:57:47.442034 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 7362
I20260811 18:57:47.443367 7466 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:57:47.550066 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 7362
2026-08-11T18:57:47Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (830 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (434 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (1644 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:57:49 dist-test-slave-h6n0 krb5kdc[7542](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:57:49 dist-test-slave-h6n0 krb5kdc[7542](info): set up 2 sockets
Aug 11 18:57:49 dist-test-slave-h6n0 krb5kdc[7542](info): commencing operation
krb5kdc: starting...
W20260811 18:57:51.012220 30146 mini_kdc.cc:121] Time spent starting KDC: real 1.339s user 0.000s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/krb5kdc/test-user.keytab.
Aug 11 18:57:51 dist-test-slave-h6n0 krb5kdc[7542](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474671, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:57:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:57:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.112.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab.
Entry for principal kudu/127.29.112.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab.
I20260811 18:57:51.299044 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.190:34403
--webserver_interface=127.29.112.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab
--principal=kudu/127.29.112.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:43515
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.190:34403
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:57:51.658433 7566 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:57:51.658795 7566 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:57:51.658962 7566 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:57:51.666287 7566 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:57:51.666388 7566 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:57:51.666486 7566 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:57:51.666618 7566 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:57:51.666676 7566 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:57:51.677655 7566 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:43515
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.112.190:34403
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab
--principal=kudu/127.29.112.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.190:34403
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.112.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:57:51.679062 7566 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:57:51.680444 7566 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:57:51.692669 7574 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:51.692619 7578 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:51.692708 7583 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:57:51.693910 7566 server_base.cc:1034] running on GCE node
I20260811 18:57:51.694742 7566 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:57:51.696122 7566 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:57:51.697356 7566 hybrid_clock.cc:648] HybridClock initialized: now 1786474671697290 us; error 40 us; skew 500 ppm
I20260811 18:57:51.697743 7566 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:57:51 dist-test-slave-h6n0 krb5kdc[7542](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474671, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:57:51.701226 7566 init.cc:377] Logged in from keytab as kudu/127.29.112.190@KRBTEST.COM (short username kudu)
I20260811 18:57:51.703080 7566 webserver.cc:460] Webserver started at http://127.29.112.190:33307/ using document root <none> and password file <none>
I20260811 18:57:51.703805 7566 fs_manager.cc:359] Metadata directory not provided
I20260811 18:57:51.703958 7566 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:57:51.704285 7566 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:57:51.707348 7566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/minicluster-data/master-0/data/instance:
uuid: "52f85404b0e9497ba37eed99016f5b49"
format_stamp: "Formatted at 2026-08-11 18:57:51 on dist-test-slave-h6n0"
server_key: "7e7757660016aff2075783d4f4095665"
server_key_iv: "06b290a81e00229c6c4c9953102506e2"
server_key_version: "encryptionkey@0"
I20260811 18:57:51.708237 7566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/minicluster-data/master-0/wal/instance:
uuid: "52f85404b0e9497ba37eed99016f5b49"
format_stamp: "Formatted at 2026-08-11 18:57:51 on dist-test-slave-h6n0"
server_key: "7e7757660016aff2075783d4f4095665"
server_key_iv: "06b290a81e00229c6c4c9953102506e2"
server_key_version: "encryptionkey@0"
I20260811 18:57:51.713586 7566 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 18:57:51.718361 7602 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:51.720427 7566 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 18:57:51.720760 7566 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/minicluster-data/master-0/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/minicluster-data/master-0/wal
uuid: "52f85404b0e9497ba37eed99016f5b49"
format_stamp: "Formatted at 2026-08-11 18:57:51 on dist-test-slave-h6n0"
server_key: "7e7757660016aff2075783d4f4095665"
server_key_iv: "06b290a81e00229c6c4c9953102506e2"
server_key_version: "encryptionkey@0"
I20260811 18:57:51.721072 7566 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:57:51.741035 7566 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:57:51.746861 7566 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:57:51.747185 7566 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:57:51.769199 7566 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.190:34403
I20260811 18:57:51.769253 7687 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.190:34403 every 8 connection(s)
I20260811 18:57:51.771232 7566 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
I20260811 18:57:51.775893 7688 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:51.780088 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 7566
I20260811 18:57:51.780335 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/minicluster-data/master-0/wal/instance
I20260811 18:57:51.781056 30146 external_mini_cluster.cc:1428] Setting key 545d7d4c2a3c85d82d7da9fede237c4f
I20260811 18:57:51.792200 7688 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52f85404b0e9497ba37eed99016f5b49: Bootstrap starting.
I20260811 18:57:51.796294 7688 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 52f85404b0e9497ba37eed99016f5b49: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:51.797387 7688 log.cc:824] T 00000000000000000000000000000000 P 52f85404b0e9497ba37eed99016f5b49: Log is configured to *not* fsync() on all Append() calls
Aug 11 18:57:51 dist-test-slave-h6n0 krb5kdc[7542](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474671, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 18:57:51.802079 7688 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52f85404b0e9497ba37eed99016f5b49: No bootstrap required, opened a new log
I20260811 18:57:51.809159 7692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:51.783406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42242 (local address 127.29.112.190:34403)
0811 18:57:51.784369 (+ 963us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:51.784386 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:51.784442 (+ 56us) server_negotiation.cc:407] Connection header received
0811 18:57:51.787152 (+ 2710us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:51.787185 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:51.787782 (+ 597us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:51.788329 (+ 547us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:51.789720 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:51.791888 (+ 2168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:51.793287 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:51.794513 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:51.794753 (+ 240us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:51.799387 (+ 4634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:51.799401 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:51.799428 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:51.799483 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:51.804311 (+ 4828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:51.805084 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:51.805094 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:51.805103 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:51.805345 (+ 242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:51.805694 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:51.805699 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:51.805703 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:51.806494 (+ 791us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:51.806836 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:51.807233 (+ 397us) server_negotiation.cc:299] Negotiation successful
0811 18:57:51.807699 (+ 466us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":692,"thread_start_us":411,"threads_started":1}
I20260811 18:57:51.813108 7688 raft_consensus.cc:348] T 00000000000000000000000000000000 P 52f85404b0e9497ba37eed99016f5b49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52f85404b0e9497ba37eed99016f5b49" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 34403 } }
I20260811 18:57:51.813469 7688 raft_consensus.cc:374] T 00000000000000000000000000000000 P 52f85404b0e9497ba37eed99016f5b49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:51.813545 7688 raft_consensus.cc:729] T 00000000000000000000000000000000 P 52f85404b0e9497ba37eed99016f5b49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52f85404b0e9497ba37eed99016f5b49, State: Initialized, Role: FOLLOWER
I20260811 18:57:51.814095 7688 consensus_queue.cc:260] T 00000000000000000000000000000000 P 52f85404b0e9497ba37eed99016f5b49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52f85404b0e9497ba37eed99016f5b49" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 34403 } }
I20260811 18:57:51.814303 7688 raft_consensus.cc:388] T 00000000000000000000000000000000 P 52f85404b0e9497ba37eed99016f5b49 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:57:51.814404 7688 raft_consensus.cc:482] T 00000000000000000000000000000000 P 52f85404b0e9497ba37eed99016f5b49 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:57:51.814580 7688 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 52f85404b0e9497ba37eed99016f5b49 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:51.817198 7688 raft_consensus.cc:504] T 00000000000000000000000000000000 P 52f85404b0e9497ba37eed99016f5b49 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52f85404b0e9497ba37eed99016f5b49" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 34403 } }
I20260811 18:57:51.817646 7688 leader_election.cc:302] T 00000000000000000000000000000000 P 52f85404b0e9497ba37eed99016f5b49 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 52f85404b0e9497ba37eed99016f5b49; no voters:
I20260811 18:57:51.818240 7688 leader_election.cc:288] T 00000000000000000000000000000000 P 52f85404b0e9497ba37eed99016f5b49 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:57:51.818403 7696 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 52f85404b0e9497ba37eed99016f5b49 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:51.819610 7696 raft_consensus.cc:686] T 00000000000000000000000000000000 P 52f85404b0e9497ba37eed99016f5b49 [term 1 LEADER]: Becoming Leader. State: Replica: 52f85404b0e9497ba37eed99016f5b49, State: Running, Role: LEADER
I20260811 18:57:51.820153 7696 consensus_queue.cc:237] T 00000000000000000000000000000000 P 52f85404b0e9497ba37eed99016f5b49 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52f85404b0e9497ba37eed99016f5b49" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 34403 } }
I20260811 18:57:51.820528 7688 sys_catalog.cc:564] T 00000000000000000000000000000000 P 52f85404b0e9497ba37eed99016f5b49 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:57:51.823390 7698 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52f85404b0e9497ba37eed99016f5b49 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 52f85404b0e9497ba37eed99016f5b49. Latest consensus state: current_term: 1 leader_uuid: "52f85404b0e9497ba37eed99016f5b49" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52f85404b0e9497ba37eed99016f5b49" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 34403 } } }
I20260811 18:57:51.823863 7698 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52f85404b0e9497ba37eed99016f5b49 [sys.catalog]: This master's current role is: LEADER
I20260811 18:57:51.823419 7697 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52f85404b0e9497ba37eed99016f5b49 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "52f85404b0e9497ba37eed99016f5b49" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52f85404b0e9497ba37eed99016f5b49" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 34403 } } }
I20260811 18:57:51.824800 7697 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52f85404b0e9497ba37eed99016f5b49 [sys.catalog]: This master's current role is: LEADER
I20260811 18:57:51.827173 7708 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:57:51.831871 7708 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:57:51.840054 7708 catalog_manager.cc:1347] Generated new cluster ID: d01df6300c39496bb3c70bb8c4eeae4b
I20260811 18:57:51.840174 7708 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:57:51.864581 7708 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:57:51.866936 7708 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:57:51.876787 7708 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 52f85404b0e9497ba37eed99016f5b49: Generated new TSK 0
I20260811 18:57:51.878141 7708 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.112.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.129.keytab.
Entry for principal kudu/127.29.112.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.129.keytab.
I20260811 18:57:51.959138 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.129:0
--local_ip_for_outbound_sockets=127.29.112.129
--webserver_interface=127.29.112.129
--webserver_port=0
--tserver_master_addrs=127.29.112.190:34403
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.129.keytab
--principal=kudu/127.29.112.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:43515
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:57:52.339953 7729 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:57:52.340247 7729 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:57:52.340410 7729 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:57:52.350165 7729 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:57:52.350311 7729 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:57:52.350522 7729 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.129
I20260811 18:57:52.362323 7729 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:43515
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.129.keytab
--principal=kudu/127.29.112.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.112.129
--webserver_port=0
--tserver_master_addrs=127.29.112.190:34403
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.129
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:57:52.363924 7729 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:57:52.365365 7729 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:57:52.381440 7738 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:52.381398 7744 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:52.381510 7740 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:57:52.382272 7729 server_base.cc:1034] running on GCE node
I20260811 18:57:52.383102 7729 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:57:52.384231 7729 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:57:52.385515 7729 hybrid_clock.cc:648] HybridClock initialized: now 1786474672385471 us; error 67 us; skew 500 ppm
I20260811 18:57:52.385895 7729 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:57:52 dist-test-slave-h6n0 krb5kdc[7542](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474672, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:57:52.390044 7729 init.cc:377] Logged in from keytab as kudu/127.29.112.129@KRBTEST.COM (short username kudu)
I20260811 18:57:52.392344 7729 webserver.cc:460] Webserver started at http://127.29.112.129:41743/ using document root <none> and password file <none>
I20260811 18:57:52.393415 7729 fs_manager.cc:359] Metadata directory not provided
I20260811 18:57:52.393836 7729 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:57:52.394302 7729 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:57:52.398978 7729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/minicluster-data/ts-0/data/instance:
uuid: "853f75d4919b4e8a9a388da775131899"
format_stamp: "Formatted at 2026-08-11 18:57:52 on dist-test-slave-h6n0"
server_key: "53d6fa52db2891555e50548f623abc82"
server_key_iv: "4972eb29d4cff7075728442404c42282"
server_key_version: "encryptionkey@0"
I20260811 18:57:52.400115 7729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/minicluster-data/ts-0/wal/instance:
uuid: "853f75d4919b4e8a9a388da775131899"
format_stamp: "Formatted at 2026-08-11 18:57:52 on dist-test-slave-h6n0"
server_key: "53d6fa52db2891555e50548f623abc82"
server_key_iv: "4972eb29d4cff7075728442404c42282"
server_key_version: "encryptionkey@0"
I20260811 18:57:52.407613 7729 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260811 18:57:52.411494 7755 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:52.412822 7729 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.005s sys 0.000s
I20260811 18:57:52.413014 7729 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/minicluster-data/ts-0/wal
uuid: "853f75d4919b4e8a9a388da775131899"
format_stamp: "Formatted at 2026-08-11 18:57:52 on dist-test-slave-h6n0"
server_key: "53d6fa52db2891555e50548f623abc82"
server_key_iv: "4972eb29d4cff7075728442404c42282"
server_key_version: "encryptionkey@0"
I20260811 18:57:52.413239 7729 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:57:52.457181 7729 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:57:52.461983 7729 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:57:52.462316 7729 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:57:52.463397 7729 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:57:52.464849 7729 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:57:52.464962 7729 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:52.465116 7729 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:57:52.465189 7729 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:52.503350 7729 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.129:38569
I20260811 18:57:52.503449 7944 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.129:38569 every 8 connection(s)
I20260811 18:57:52.504739 7729 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/minicluster-data/ts-0/data/info.pb
I20260811 18:57:52.512562 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 7729
I20260811 18:57:52.512832 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/minicluster-data/ts-0/wal/instance
I20260811 18:57:52.513686 30146 external_mini_cluster.cc:1428] Setting key 79fcd078f102bb7f747a7ea5481096a8
Aug 11 18:57:52 dist-test-slave-h6n0 krb5kdc[7542](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474672, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.129@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 18:57:52.528914 7955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:52.507574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.129:54187 (local address 127.29.112.190:34403)
0811 18:57:52.508034 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:52.508039 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:52.508984 (+ 945us) server_negotiation.cc:407] Connection header received
0811 18:57:52.511068 (+ 2084us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:52.511077 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:52.511184 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:52.511284 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:52.513325 (+ 2041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:52.514116 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:52.516404 (+ 2288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:52.517083 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:52.517323 (+ 240us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:52.522336 (+ 5013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:52.522342 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:52.522345 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:52.522376 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:52.525060 (+ 2684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:52.526163 (+ 1103us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:52.526167 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:52.526169 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:52.526372 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:52.527046 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:52.527050 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:52.527052 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:52.527374 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:52.527492 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:52.528488 (+ 996us) server_negotiation.cc:299] Negotiation successful
0811 18:57:52.528660 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":231,"threads_started":1}
I20260811 18:57:52.530577 7956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:52.508094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:34403 (local address 127.29.112.129:54187)
0811 18:57:52.508825 (+ 731us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:52.508866 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:52.510755 (+ 1889us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:52.511520 (+ 765us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:52.511528 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:52.512221 (+ 693us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:57:52.513134 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:52.513165 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:52.514291 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:52.514297 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:52.516224 (+ 1927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:52.516241 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:52.517413 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:52.517424 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:52.517849 (+ 425us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:52.518683 (+ 834us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:57:52.518720 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:57:52.522104 (+ 3384us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:57:52.525235 (+ 3131us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:52.525248 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:52.525267 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:52.526023 (+ 756us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:52.526499 (+ 476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:52.526507 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:52.526513 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:52.526920 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:52.527523 (+ 603us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:57:52.527535 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:57:52.528020 (+ 485us) client_negotiation.cc:769] Sending connection context
0811 18:57:52.528378 (+ 358us) client_negotiation.cc:240] Negotiation successful
0811 18:57:52.528992 (+ 614us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":314,"threads_started":1}
I20260811 18:57:52.532220 7947 heartbeater.cc:347] Connected to a master server at 127.29.112.190:34403
I20260811 18:57:52.532557 7947 heartbeater.cc:464] Registering TS with master...
I20260811 18:57:52.533275 7947 heartbeater.cc:511] Master 127.29.112.190:34403 requested a full tablet report, sending...
I20260811 18:57:52.535480 7631 ts_manager.cc:194] Registered new tserver with Master: 853f75d4919b4e8a9a388da775131899 (127.29.112.129:38569)
I20260811 18:57:52.536903 7631 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.129@KRBTEST.COM'} at 127.29.112.129:54187
WARNING: no policy specified for kudu/127.29.112.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.130.keytab.
Entry for principal kudu/127.29.112.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.130.keytab.
I20260811 18:57:52.573935 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.130:0
--local_ip_for_outbound_sockets=127.29.112.130
--webserver_interface=127.29.112.130
--webserver_port=0
--tserver_master_addrs=127.29.112.190:34403
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.130.keytab
--principal=kudu/127.29.112.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:43515
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:57:52.925383 7959 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:57:52.925733 7959 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:57:52.925905 7959 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:57:52.934234 7959 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:57:52.934351 7959 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:57:52.934468 7959 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.130
I20260811 18:57:52.944525 7959 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:43515
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.130.keytab
--principal=kudu/127.29.112.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.112.130
--webserver_port=0
--tserver_master_addrs=127.29.112.190:34403
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.130
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:57:52.945966 7959 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:57:52.947961 7959 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:57:52.962191 7972 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:52.962224 7970 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:52.962457 7969 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:57:52.963356 7959 server_base.cc:1034] running on GCE node
I20260811 18:57:52.963997 7959 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:57:52.964944 7959 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:57:52.966169 7959 hybrid_clock.cc:648] HybridClock initialized: now 1786474672966118 us; error 51 us; skew 500 ppm
I20260811 18:57:52.966475 7959 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:57:52 dist-test-slave-h6n0 krb5kdc[7542](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474672, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:57:52.969693 7959 init.cc:377] Logged in from keytab as kudu/127.29.112.130@KRBTEST.COM (short username kudu)
I20260811 18:57:52.971191 7959 webserver.cc:460] Webserver started at http://127.29.112.130:33953/ using document root <none> and password file <none>
I20260811 18:57:52.971868 7959 fs_manager.cc:359] Metadata directory not provided
I20260811 18:57:52.971997 7959 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:57:52.972311 7959 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:57:52.975560 7959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/minicluster-data/ts-1/data/instance:
uuid: "7b4d883bc46844e1ba4729bc10fbae84"
format_stamp: "Formatted at 2026-08-11 18:57:52 on dist-test-slave-h6n0"
server_key: "f6e7a04f1f9be9c3071c114ebe1c77d1"
server_key_iv: "e2cecdf57e312cf8515cb14de13dc101"
server_key_version: "encryptionkey@0"
I20260811 18:57:52.976961 7959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/minicluster-data/ts-1/wal/instance:
uuid: "7b4d883bc46844e1ba4729bc10fbae84"
format_stamp: "Formatted at 2026-08-11 18:57:52 on dist-test-slave-h6n0"
server_key: "f6e7a04f1f9be9c3071c114ebe1c77d1"
server_key_iv: "e2cecdf57e312cf8515cb14de13dc101"
server_key_version: "encryptionkey@0"
I20260811 18:57:52.984392 7959 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.004s sys 0.004s
I20260811 18:57:52.988925 7984 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:52.990377 7959 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 18:57:52.990617 7959 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/minicluster-data/ts-1/wal
uuid: "7b4d883bc46844e1ba4729bc10fbae84"
format_stamp: "Formatted at 2026-08-11 18:57:52 on dist-test-slave-h6n0"
server_key: "f6e7a04f1f9be9c3071c114ebe1c77d1"
server_key_iv: "e2cecdf57e312cf8515cb14de13dc101"
server_key_version: "encryptionkey@0"
I20260811 18:57:52.990883 7959 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:57:53.006191 7959 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:57:53.011241 7959 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:57:53.011564 7959 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:57:53.012583 7959 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:57:53.014017 7959 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:57:53.014099 7959 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:53.014281 7959 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:57:53.014341 7959 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:53.053701 7959 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.130:45403
I20260811 18:57:53.053769 8207 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.130:45403 every 8 connection(s)
I20260811 18:57:53.055351 7959 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/minicluster-data/ts-1/data/info.pb
I20260811 18:57:53.062033 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 7959
I20260811 18:57:53.062263 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/minicluster-data/ts-1/wal/instance
I20260811 18:57:53.062798 30146 external_mini_cluster.cc:1428] Setting key dccd8a6535b1c3e92d363b6494365dfb
Aug 11 18:57:53 dist-test-slave-h6n0 krb5kdc[7542](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474672, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.130@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 18:57:53.079860 8217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:53.058205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.130:57213 (local address 127.29.112.190:34403)
0811 18:57:53.058905 (+ 700us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:53.058914 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:53.060047 (+ 1133us) server_negotiation.cc:407] Connection header received
0811 18:57:53.061965 (+ 1918us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:53.061969 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:53.062043 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:53.062208 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:53.064083 (+ 1875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:53.065022 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:53.066717 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:53.067529 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:53.067673 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:53.073033 (+ 5360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:53.073042 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:53.073050 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:53.073101 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:53.076562 (+ 3461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:53.077451 (+ 889us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:53.077456 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:53.077459 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:53.077582 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:53.078026 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:53.078030 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:53.078033 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:53.078311 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:53.078466 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:53.079253 (+ 787us) server_negotiation.cc:299] Negotiation successful
0811 18:57:53.079479 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":517,"thread_start_us":335,"threads_started":1}
I20260811 18:57:53.080852 8218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:53.058675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:34403 (local address 127.29.112.130:57213)
0811 18:57:53.059864 (+ 1189us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:53.059921 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:53.061630 (+ 1709us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:53.062450 (+ 820us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:53.062458 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:53.063113 (+ 655us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:57:53.063881 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:53.063908 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:53.065200 (+ 1292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:53.065204 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:53.066502 (+ 1298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:53.066515 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:53.067783 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:53.067787 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:53.068182 (+ 395us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:53.069276 (+ 1094us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:57:53.069308 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:57:53.072668 (+ 3360us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:57:53.076757 (+ 4089us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:53.076767 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:53.076782 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:53.077306 (+ 524us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:53.077693 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:53.077698 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:53.077701 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:53.077899 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:53.078442 (+ 543us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:57:53.078460 (+ 18us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:57:53.078879 (+ 419us) client_negotiation.cc:769] Sending connection context
0811 18:57:53.079148 (+ 269us) client_negotiation.cc:240] Negotiation successful
0811 18:57:53.079715 (+ 567us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":890,"thread_start_us":612,"threads_started":1}
I20260811 18:57:53.082315 8211 heartbeater.cc:347] Connected to a master server at 127.29.112.190:34403
I20260811 18:57:53.082731 8211 heartbeater.cc:464] Registering TS with master...
I20260811 18:57:53.083644 8211 heartbeater.cc:511] Master 127.29.112.190:34403 requested a full tablet report, sending...
I20260811 18:57:53.085376 7631 ts_manager.cc:194] Registered new tserver with Master: 7b4d883bc46844e1ba4729bc10fbae84 (127.29.112.130:45403)
I20260811 18:57:53.086709 7631 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.130@KRBTEST.COM'} at 127.29.112.130:57213
WARNING: no policy specified for kudu/127.29.112.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.131.keytab.
Entry for principal kudu/127.29.112.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.131.keytab.
I20260811 18:57:53.122176 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.131:0
--local_ip_for_outbound_sockets=127.29.112.131
--webserver_interface=127.29.112.131
--webserver_port=0
--tserver_master_addrs=127.29.112.190:34403
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.131.keytab
--principal=kudu/127.29.112.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:43515
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:57:53.470985 8223 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:57:53.471287 8223 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:57:53.471385 8223 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:57:53.478806 8223 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:57:53.478976 8223 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:57:53.479166 8223 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.131
I20260811 18:57:53.488973 8223 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:43515
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.131.keytab
--principal=kudu/127.29.112.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.112.131
--webserver_port=0
--tserver_master_addrs=127.29.112.190:34403
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.131
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:57:53.490302 8223 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:57:53.491638 8223 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:57:53.505412 8241 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:53.505934 8242 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:53.507187 8245 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:57:53.507788 8223 server_base.cc:1034] running on GCE node
I20260811 18:57:53.508522 8223 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:57:53.509732 8223 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:57:53.510949 8223 hybrid_clock.cc:648] HybridClock initialized: now 1786474673510906 us; error 40 us; skew 500 ppm
I20260811 18:57:53.511267 8223 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:57:53 dist-test-slave-h6n0 krb5kdc[7542](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474673, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:57:53.514719 8223 init.cc:377] Logged in from keytab as kudu/127.29.112.131@KRBTEST.COM (short username kudu)
I20260811 18:57:53.516249 8223 webserver.cc:460] Webserver started at http://127.29.112.131:38031/ using document root <none> and password file <none>
I20260811 18:57:53.516983 8223 fs_manager.cc:359] Metadata directory not provided
I20260811 18:57:53.517113 8223 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:57:53.517459 8223 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:57:53.520390 8223 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/minicluster-data/ts-2/data/instance:
uuid: "82a8db94d26c4087add5cbf0615e1046"
format_stamp: "Formatted at 2026-08-11 18:57:53 on dist-test-slave-h6n0"
server_key: "5447d8bf0e4d672d01138ac47ea79a51"
server_key_iv: "44605875b40d13484630c816346d062d"
server_key_version: "encryptionkey@0"
I20260811 18:57:53.521345 8223 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/minicluster-data/ts-2/wal/instance:
uuid: "82a8db94d26c4087add5cbf0615e1046"
format_stamp: "Formatted at 2026-08-11 18:57:53 on dist-test-slave-h6n0"
server_key: "5447d8bf0e4d672d01138ac47ea79a51"
server_key_iv: "44605875b40d13484630c816346d062d"
server_key_version: "encryptionkey@0"
I20260811 18:57:53.526947 8223 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 18:57:53.530414 8254 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:53.531725 8223 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:57:53.531913 8223 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/minicluster-data/ts-2/wal
uuid: "82a8db94d26c4087add5cbf0615e1046"
format_stamp: "Formatted at 2026-08-11 18:57:53 on dist-test-slave-h6n0"
server_key: "5447d8bf0e4d672d01138ac47ea79a51"
server_key_iv: "44605875b40d13484630c816346d062d"
server_key_version: "encryptionkey@0"
I20260811 18:57:53.532135 8223 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:57:53.540966 7947 heartbeater.cc:503] Master 127.29.112.190:34403 was elected leader, sending a full tablet report...
I20260811 18:57:53.562464 8223 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:57:53.566851 8223 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:57:53.567159 8223 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:57:53.568213 8223 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:57:53.569600 8223 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:57:53.569705 8223 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:53.569854 8223 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:57:53.569933 8223 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:53.610522 8223 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.131:45195
I20260811 18:57:53.610638 8453 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.131:45195 every 8 connection(s)
I20260811 18:57:53.611992 8223 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/minicluster-data/ts-2/data/info.pb
I20260811 18:57:53.621583 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 8223
I20260811 18:57:53.621801 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/minicluster-data/ts-2/wal/instance
I20260811 18:57:53.622280 30146 external_mini_cluster.cc:1428] Setting key 7e6df29524674d072b39a0ee548db07b
Aug 11 18:57:53 dist-test-slave-h6n0 krb5kdc[7542](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474673, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.131@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 18:57:53.634752 8456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:53.615016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.131:57503 (local address 127.29.112.190:34403)
0811 18:57:53.615605 (+ 589us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:53.615611 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:53.616834 (+ 1223us) server_negotiation.cc:407] Connection header received
0811 18:57:53.619062 (+ 2228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:53.619070 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:53.619186 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:53.619329 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:53.621206 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:53.622286 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:53.624538 (+ 2252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:53.625033 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:53.625204 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:53.628997 (+ 3793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:53.629002 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:53.629005 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:53.629038 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:53.631681 (+ 2643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:53.632593 (+ 912us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:53.632597 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:53.632599 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:53.632727 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:53.633167 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:53.633171 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:53.633173 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:53.633434 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:53.633555 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:53.634222 (+ 667us) server_negotiation.cc:299] Negotiation successful
0811 18:57:53.634381 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":462,"thread_start_us":314,"threads_started":1}
I20260811 18:57:53.635866 8457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:53.615717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:34403 (local address 127.29.112.131:57503)
0811 18:57:53.616636 (+ 919us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:53.616680 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:53.618740 (+ 2060us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:53.619616 (+ 876us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:53.619625 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:53.620228 (+ 603us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:57:53.620982 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:53.621005 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:53.622457 (+ 1452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:53.622465 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:53.624365 (+ 1900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:53.624380 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:53.625274 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:53.625279 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:53.625623 (+ 344us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:53.626380 (+ 757us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:57:53.626422 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:57:53.628797 (+ 2375us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:57:53.631841 (+ 3044us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:53.631849 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:53.631861 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:53.632460 (+ 599us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:53.632841 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:53.632844 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:53.632847 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:53.633055 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:53.633547 (+ 492us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:57:53.633553 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:57:53.633892 (+ 339us) client_negotiation.cc:769] Sending connection context
0811 18:57:53.634129 (+ 237us) client_negotiation.cc:240] Negotiation successful
0811 18:57:53.634611 (+ 482us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":637,"thread_start_us":370,"threads_started":1}
I20260811 18:57:53.637430 8454 heartbeater.cc:347] Connected to a master server at 127.29.112.190:34403
I20260811 18:57:53.637771 8454 heartbeater.cc:464] Registering TS with master...
I20260811 18:57:53.638502 8454 heartbeater.cc:511] Master 127.29.112.190:34403 requested a full tablet report, sending...
I20260811 18:57:53.640062 7631 ts_manager.cc:194] Registered new tserver with Master: 82a8db94d26c4087add5cbf0615e1046 (127.29.112.131:45195)
I20260811 18:57:53.640880 7631 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.131@KRBTEST.COM'} at 127.29.112.131:57503
I20260811 18:57:53.650391 30146 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/krb5kdc/test-admin.keytab.
Aug 11 18:57:53 dist-test-slave-h6n0 krb5kdc[7542](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474673, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:57:53.685104 30146 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260811 18:57:53.690294 8467 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:57:53.687904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55798 (local address 127.29.112.129:38569)
0811 18:57:53.688524 (+ 620us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:53.688532 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:53.688559 (+ 27us) server_negotiation.cc:407] Connection header received
0811 18:57:53.688634 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:53.688640 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:53.688836 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:53.688967 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:53.689804 (+ 837us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55798: BlockingRecv error: recv got EOF from 127.0.0.1:55798 (error 108)
Metrics: {"server-negotiator.queue_time_us":473,"thread_start_us":321,"threads_started":1}
W20260811 18:57:53.690467 30146 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/krb5kdc/krb5cc' of type FILE
Aug 11 18:57:53 dist-test-slave-h6n0 krb5kdc[7542](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474673, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:57:53.692122 30146 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 18:57:53 dist-test-slave-h6n0 krb5kdc[7542](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474673, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:57:53.693573 30146 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 18:57:53 dist-test-slave-h6n0 krb5kdc[7542](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474673, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.129@KRBTEST.COM
I20260811 18:57:53.713187 8467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:53.695364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55812 (local address 127.29.112.129:38569)
0811 18:57:53.695611 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:53.695620 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:53.695891 (+ 271us) server_negotiation.cc:407] Connection header received
0811 18:57:53.696128 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:53.696133 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:53.696204 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:53.696333 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:53.697516 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:53.698738 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:53.700214 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:53.700857 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:53.701021 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:53.705884 (+ 4863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:53.705896 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:53.705910 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:53.705957 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:53.709783 (+ 3826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:53.710520 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:53.710577 (+ 57us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:53.710582 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:53.710737 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:53.711303 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:53.711310 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:53.711314 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:53.711886 (+ 572us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:53.712315 (+ 429us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:53.712617 (+ 302us) server_negotiation.cc:299] Negotiation successful
0811 18:57:53.712859 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
W20260811 18:57:53.720516 8467 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:57:53.718194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55826 (local address 127.29.112.129:38569)
0811 18:57:53.718391 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:53.718396 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:53.719079 (+ 683us) server_negotiation.cc:407] Connection header received
0811 18:57:53.719278 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:53.719282 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:53.719340 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:53.719446 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:53.720238 (+ 792us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55826: BlockingRecv error: recv got EOF from 127.0.0.1:55826 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260811 18:57:53.721104 30146 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/krb5kdc/krb5cc)
Aug 11 18:57:53 dist-test-slave-h6n0 krb5kdc[7542](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474673, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:57:53.723255 30146 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 18:57:53 dist-test-slave-h6n0 krb5kdc[7542](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474673, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:57:53.724581 30146 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 18:57:53 dist-test-slave-h6n0 krb5kdc[7542](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474673, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.129@KRBTEST.COM
I20260811 18:57:53.741869 8467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:53.726188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55832 (local address 127.29.112.129:38569)
0811 18:57:53.726408 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:53.726415 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:53.726739 (+ 324us) server_negotiation.cc:407] Connection header received
0811 18:57:53.726844 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:53.726847 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:53.726905 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:53.726988 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:53.728412 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:53.729123 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:53.730375 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:53.730985 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:53.731112 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:53.736215 (+ 5103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:53.736220 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:53.736224 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:53.736263 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:53.739244 (+ 2981us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:53.739961 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:53.739965 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:53.739967 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:53.740093 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:53.740436 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:53.740439 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:53.740441 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:53.740731 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:53.740935 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:53.741425 (+ 490us) server_negotiation.cc:299] Negotiation successful
0811 18:57:53.741608 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
W20260811 18:57:53.746600 8467 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:57:53.744801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55848 (local address 127.29.112.129:38569)
0811 18:57:53.745046 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:53.745052 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:53.745170 (+ 118us) server_negotiation.cc:407] Connection header received
0811 18:57:53.745366 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:53.745371 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:53.745447 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:53.745578 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:53.746285 (+ 707us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55848: BlockingRecv error: recv got EOF from 127.0.0.1:55848 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
W20260811 18:57:53.746939 30146 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474615205622-30146-0/krb5kdc/krb5cc' of type FILE
Aug 11 18:57:53 dist-test-slave-h6n0 krb5kdc[7542](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474673, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:57:53.748636 30146 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 18:57:53 dist-test-slave-h6n0 krb5kdc[7542](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474673, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:57:53.750110 30146 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 18:57:53 dist-test-slave-h6n0 krb5kdc[7542](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474673, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.129@KRBTEST.COM
I20260811 18:57:53.767005 8467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:53.751502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55856 (local address 127.29.112.129:38569)
0811 18:57:53.751722 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:53.751729 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:53.752002 (+ 273us) server_negotiation.cc:407] Connection header received
0811 18:57:53.752178 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:53.752183 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:53.752243 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:53.752342 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:53.753250 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:53.753962 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:53.755220 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:53.755656 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:53.755791 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:53.759594 (+ 3803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:53.759598 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:53.759602 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:53.759634 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:53.763432 (+ 3798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:53.764290 (+ 858us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:53.764297 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:53.764299 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:53.764510 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:53.764966 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:53.764971 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:53.764973 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:53.765413 (+ 440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:53.765765 (+ 352us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:53.766041 (+ 276us) server_negotiation.cc:299] Negotiation successful
0811 18:57:53.766681 (+ 640us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 18:57:53.769176 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 7729
I20260811 18:57:53.783704 8467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:53.769983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55870 (local address 127.29.112.129:38569)
0811 18:57:53.770183 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:53.770191 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:53.770623 (+ 432us) server_negotiation.cc:407] Connection header received
0811 18:57:53.770854 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:53.770860 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:53.770936 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:53.771338 (+ 402us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:53.772572 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:53.773367 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:53.774883 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:53.775391 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:53.775576 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:53.778096 (+ 2520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:53.778101 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:53.778106 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:53.778141 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:53.781054 (+ 2913us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:53.781703 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:53.781710 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:53.781713 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:53.781854 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:53.782237 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:53.782242 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:53.782245 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:53.782519 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:53.782855 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:53.783180 (+ 325us) server_negotiation.cc:299] Negotiation successful
0811 18:57:53.783385 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 18:57:53.784307 7937 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:57:53.875494 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 7729
I20260811 18:57:53.902478 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 7959
Aug 11 18:57:53 dist-test-slave-h6n0 krb5kdc[7542](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474673, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.130@KRBTEST.COM
I20260811 18:57:53.920912 8495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:53.903886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53746 (local address 127.29.112.130:45403)
0811 18:57:53.904634 (+ 748us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:53.904643 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:53.904678 (+ 35us) server_negotiation.cc:407] Connection header received
0811 18:57:53.904757 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:53.904768 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:53.905053 (+ 285us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:53.905225 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:53.907068 (+ 1843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:53.908278 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:53.909621 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:53.910326 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:53.910450 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:53.914815 (+ 4365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:53.914831 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:53.914848 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:53.914887 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:53.917919 (+ 3032us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:53.918679 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:53.918686 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:53.918689 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:53.918813 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:53.919235 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:53.919239 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:53.919241 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:53.919606 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:53.919941 (+ 335us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:53.920265 (+ 324us) server_negotiation.cc:299] Negotiation successful
0811 18:57:53.920569 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":568,"thread_start_us":353,"threads_started":1}
I20260811 18:57:53.922503 8198 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:57:54.009771 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 7959
I20260811 18:57:54.037428 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 8223
Aug 11 18:57:54 dist-test-slave-h6n0 krb5kdc[7542](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474673, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.131@KRBTEST.COM
I20260811 18:57:54.056231 8500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:54.038468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33344 (local address 127.29.112.131:45195)
0811 18:57:54.039049 (+ 581us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:54.039056 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:54.039085 (+ 29us) server_negotiation.cc:407] Connection header received
0811 18:57:54.039145 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:54.039151 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:54.039402 (+ 251us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:54.039581 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:54.040671 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:54.041635 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:54.043041 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:54.043611 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:54.043752 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:54.049119 (+ 5367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:54.049133 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:54.049149 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:54.049187 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:54.052686 (+ 3499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:54.053551 (+ 865us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:54.053558 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:54.053560 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:54.053692 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:54.054125 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:54.054129 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:54.054131 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:54.054523 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:54.055169 (+ 646us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:54.055437 (+ 268us) server_negotiation.cc:299] Negotiation successful
0811 18:57:54.055909 (+ 472us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":444,"thread_start_us":313,"threads_started":1}
I20260811 18:57:54.058199 8441 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:57:54.147172 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 8223
I20260811 18:57:54.169771 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 7566
I20260811 18:57:54.171173 7678 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:57:54.278211 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 7566
2026-08-11T18:57:54Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (4656 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:57:54 dist-test-slave-h6n0 krb5kdc[8508](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:57:54 dist-test-slave-h6n0 krb5kdc[8508](info): set up 2 sockets
Aug 11 18:57:54 dist-test-slave-h6n0 krb5kdc[8508](info): commencing operation
krb5kdc: starting...
W20260811 18:57:55.656466 30146 mini_kdc.cc:121] Time spent starting KDC: real 1.328s user 0.001s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/krb5kdc/test-user.keytab.
Aug 11 18:57:55 dist-test-slave-h6n0 krb5kdc[8508](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474675, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:57:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:57:55Z Disabled control of system clock
WARNING: no policy specified for oryx/127.29.112.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.112.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.112.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/krb5kdc/oryx_127.29.112.190.keytab.
Entry for principal oryx/127.29.112.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/krb5kdc/oryx_127.29.112.190.keytab.
I20260811 18:57:55.934991 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.190:37199
--webserver_interface=127.29.112.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/krb5kdc/oryx_127.29.112.190.keytab
--principal=oryx/127.29.112.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:43681
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.190:37199
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:57:56.288973 8537 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:57:56.289392 8537 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:57:56.289573 8537 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:57:56.297502 8537 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:57:56.297710 8537 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:57:56.297839 8537 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:57:56.297952 8537 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:57:56.298048 8537 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:57:56.310669 8537 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:43681
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.112.190:37199
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/krb5kdc/oryx_127.29.112.190.keytab
--principal=oryx/127.29.112.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.190:37199
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.112.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:57:56.312404 8537 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:57:56.314046 8537 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:57:56.328163 8547 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:56.328310 8552 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:56.328120 8549 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:57:56.329537 8537 server_base.cc:1034] running on GCE node
I20260811 18:57:56.330448 8537 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:57:56.332068 8537 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:57:56.333359 8537 hybrid_clock.cc:648] HybridClock initialized: now 1786474676333242 us; error 86 us; skew 500 ppm
I20260811 18:57:56.333808 8537 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:57:56 dist-test-slave-h6n0 krb5kdc[8508](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474676, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.112.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:57:56.341566 8537 init.cc:377] Logged in from keytab as oryx/127.29.112.190@KRBTEST.COM (short username oryx)
I20260811 18:57:56.343874 8537 webserver.cc:460] Webserver started at http://127.29.112.190:41109/ using document root <none> and password file <none>
I20260811 18:57:56.345018 8537 fs_manager.cc:359] Metadata directory not provided
I20260811 18:57:56.345220 8537 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:57:56.345681 8537 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:57:56.350143 8537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/master-0/data/instance:
uuid: "b994bc87dbdf4d93a9b50500ad3f2aca"
format_stamp: "Formatted at 2026-08-11 18:57:56 on dist-test-slave-h6n0"
server_key: "26f3d6a50fd1c448e0ef2931dca1da81"
server_key_iv: "a25c256ebacbd05faf8930235981b445"
server_key_version: "encryptionkey@0"
I20260811 18:57:56.351540 8537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/master-0/wal/instance:
uuid: "b994bc87dbdf4d93a9b50500ad3f2aca"
format_stamp: "Formatted at 2026-08-11 18:57:56 on dist-test-slave-h6n0"
server_key: "26f3d6a50fd1c448e0ef2931dca1da81"
server_key_iv: "a25c256ebacbd05faf8930235981b445"
server_key_version: "encryptionkey@0"
I20260811 18:57:56.357839 8537 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260811 18:57:56.362870 8566 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:56.364835 8537 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 18:57:56.365115 8537 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/master-0/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/master-0/wal
uuid: "b994bc87dbdf4d93a9b50500ad3f2aca"
format_stamp: "Formatted at 2026-08-11 18:57:56 on dist-test-slave-h6n0"
server_key: "26f3d6a50fd1c448e0ef2931dca1da81"
server_key_iv: "a25c256ebacbd05faf8930235981b445"
server_key_version: "encryptionkey@0"
I20260811 18:57:56.365463 8537 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:57:56.401206 8537 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:57:56.405882 8537 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:57:56.406241 8537 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:57:56.430622 8537 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.190:37199
I20260811 18:57:56.430747 8647 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.190:37199 every 8 connection(s)
I20260811 18:57:56.432112 8537 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
I20260811 18:57:56.435052 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 8537
I20260811 18:57:56.435303 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/master-0/wal/instance
I20260811 18:57:56.436159 30146 external_mini_cluster.cc:1428] Setting key 0cd9fc8f25fbee62cac5031bf68bf0ab
I20260811 18:57:56.436316 8650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:56.452307 8650 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b994bc87dbdf4d93a9b50500ad3f2aca: Bootstrap starting.
Aug 11 18:57:56 dist-test-slave-h6n0 krb5kdc[8508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474675, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM
I20260811 18:57:56.455749 8650 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b994bc87dbdf4d93a9b50500ad3f2aca: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:56.456923 8650 log.cc:824] T 00000000000000000000000000000000 P b994bc87dbdf4d93a9b50500ad3f2aca: Log is configured to *not* fsync() on all Append() calls
I20260811 18:57:56.460026 8650 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b994bc87dbdf4d93a9b50500ad3f2aca: No bootstrap required, opened a new log
I20260811 18:57:56.460960 8655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:56.438270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40512 (local address 127.29.112.190:37199)
0811 18:57:56.439100 (+ 830us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:56.439111 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:56.439154 (+ 43us) server_negotiation.cc:407] Connection header received
0811 18:57:56.441452 (+ 2298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:56.441474 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:56.441994 (+ 520us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:56.442368 (+ 374us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:56.444514 (+ 2146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:56.445812 (+ 1298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:56.447787 (+ 1975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:56.448550 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:56.448698 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:56.453792 (+ 5094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:56.453800 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:56.453813 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:56.453858 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:56.457027 (+ 3169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:56.457685 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:56.457692 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:56.457697 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:56.457866 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:56.458275 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:56.458279 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:56.458282 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:56.458845 (+ 563us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:56.459169 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:56.459600 (+ 431us) server_negotiation.cc:299] Negotiation successful
0811 18:57:56.460008 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":631,"thread_start_us":388,"threads_started":1}
I20260811 18:57:56.468878 8650 raft_consensus.cc:348] T 00000000000000000000000000000000 P b994bc87dbdf4d93a9b50500ad3f2aca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b994bc87dbdf4d93a9b50500ad3f2aca" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 37199 } }
I20260811 18:57:56.469249 8650 raft_consensus.cc:374] T 00000000000000000000000000000000 P b994bc87dbdf4d93a9b50500ad3f2aca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:56.469363 8650 raft_consensus.cc:729] T 00000000000000000000000000000000 P b994bc87dbdf4d93a9b50500ad3f2aca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b994bc87dbdf4d93a9b50500ad3f2aca, State: Initialized, Role: FOLLOWER
I20260811 18:57:56.470389 8650 consensus_queue.cc:260] T 00000000000000000000000000000000 P b994bc87dbdf4d93a9b50500ad3f2aca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b994bc87dbdf4d93a9b50500ad3f2aca" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 37199 } }
I20260811 18:57:56.470835 8660 raft_consensus.cc:482] T 00000000000000000000000000000000 P b994bc87dbdf4d93a9b50500ad3f2aca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:56.471136 8660 raft_consensus.cc:504] T 00000000000000000000000000000000 P b994bc87dbdf4d93a9b50500ad3f2aca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b994bc87dbdf4d93a9b50500ad3f2aca" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 37199 } }
I20260811 18:57:56.470808 8650 raft_consensus.cc:388] T 00000000000000000000000000000000 P b994bc87dbdf4d93a9b50500ad3f2aca [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:57:56.471808 8650 raft_consensus.cc:482] T 00000000000000000000000000000000 P b994bc87dbdf4d93a9b50500ad3f2aca [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:57:56.471776 8660 leader_election.cc:302] T 00000000000000000000000000000000 P b994bc87dbdf4d93a9b50500ad3f2aca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b994bc87dbdf4d93a9b50500ad3f2aca; no voters:
I20260811 18:57:56.471997 8650 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b994bc87dbdf4d93a9b50500ad3f2aca [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:56.472688 8660 leader_election.cc:288] T 00000000000000000000000000000000 P b994bc87dbdf4d93a9b50500ad3f2aca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers
I20260811 18:57:56.474735 8650 raft_consensus.cc:504] T 00000000000000000000000000000000 P b994bc87dbdf4d93a9b50500ad3f2aca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b994bc87dbdf4d93a9b50500ad3f2aca" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 37199 } }
I20260811 18:57:56.475383 8650 leader_election.cc:302] T 00000000000000000000000000000000 P b994bc87dbdf4d93a9b50500ad3f2aca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b994bc87dbdf4d93a9b50500ad3f2aca; no voters:
I20260811 18:57:56.475379 8662 raft_consensus.cc:2741] T 00000000000000000000000000000000 P b994bc87dbdf4d93a9b50500ad3f2aca [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260811 18:57:56.475922 8650 leader_election.cc:288] T 00000000000000000000000000000000 P b994bc87dbdf4d93a9b50500ad3f2aca [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:57:56.476048 8660 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b994bc87dbdf4d93a9b50500ad3f2aca [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:56.477126 8660 raft_consensus.cc:686] T 00000000000000000000000000000000 P b994bc87dbdf4d93a9b50500ad3f2aca [term 1 LEADER]: Becoming Leader. State: Replica: b994bc87dbdf4d93a9b50500ad3f2aca, State: Running, Role: LEADER
I20260811 18:57:56.477717 8660 consensus_queue.cc:237] T 00000000000000000000000000000000 P b994bc87dbdf4d93a9b50500ad3f2aca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b994bc87dbdf4d93a9b50500ad3f2aca" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 37199 } }
I20260811 18:57:56.478434 8650 sys_catalog.cc:564] T 00000000000000000000000000000000 P b994bc87dbdf4d93a9b50500ad3f2aca [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:57:56.480090 8662 sys_catalog.cc:455] T 00000000000000000000000000000000 P b994bc87dbdf4d93a9b50500ad3f2aca [sys.catalog]: SysCatalogTable state changed. Reason: New leader b994bc87dbdf4d93a9b50500ad3f2aca. Latest consensus state: current_term: 1 leader_uuid: "b994bc87dbdf4d93a9b50500ad3f2aca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b994bc87dbdf4d93a9b50500ad3f2aca" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 37199 } } }
I20260811 18:57:56.480079 8665 sys_catalog.cc:455] T 00000000000000000000000000000000 P b994bc87dbdf4d93a9b50500ad3f2aca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b994bc87dbdf4d93a9b50500ad3f2aca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b994bc87dbdf4d93a9b50500ad3f2aca" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 37199 } } }
I20260811 18:57:56.480671 8662 sys_catalog.cc:458] T 00000000000000000000000000000000 P b994bc87dbdf4d93a9b50500ad3f2aca [sys.catalog]: This master's current role is: LEADER
I20260811 18:57:56.480710 8665 sys_catalog.cc:458] T 00000000000000000000000000000000 P b994bc87dbdf4d93a9b50500ad3f2aca [sys.catalog]: This master's current role is: LEADER
I20260811 18:57:56.482313 8671 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:57:56.486626 8671 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:57:56.495090 8671 catalog_manager.cc:1347] Generated new cluster ID: ddf73710ba7c49fbb2a6a28cae586e9c
I20260811 18:57:56.495198 8671 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:57:56.522861 8671 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:57:56.524209 8671 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:57:56.536531 8671 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b994bc87dbdf4d93a9b50500ad3f2aca: Generated new TSK 0
I20260811 18:57:56.537434 8671 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.29.112.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.112.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.112.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/krb5kdc/oryx_127.29.112.129.keytab.
Entry for principal oryx/127.29.112.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/krb5kdc/oryx_127.29.112.129.keytab.
I20260811 18:57:56.610081 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.129:0
--local_ip_for_outbound_sockets=127.29.112.129
--webserver_interface=127.29.112.129
--webserver_port=0
--tserver_master_addrs=127.29.112.190:37199
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/krb5kdc/oryx_127.29.112.129.keytab
--principal=oryx/127.29.112.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:43681
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:57:56.977141 8697 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:57:56.977483 8697 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:57:56.977658 8697 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:57:56.985088 8697 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:57:56.985244 8697 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:57:56.985416 8697 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.129
I20260811 18:57:56.995266 8697 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:43681
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/krb5kdc/oryx_127.29.112.129.keytab
--principal=oryx/127.29.112.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.112.129
--webserver_port=0
--tserver_master_addrs=127.29.112.190:37199
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.129
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:57:56.996506 8697 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:57:56.997778 8697 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:57:57.011620 8716 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:57.011744 8717 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:57.011904 8722 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:57:57.013211 8697 server_base.cc:1034] running on GCE node
I20260811 18:57:57.013816 8697 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:57:57.014825 8697 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:57:57.016047 8697 hybrid_clock.cc:648] HybridClock initialized: now 1786474677015990 us; error 44 us; skew 500 ppm
I20260811 18:57:57.016326 8697 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:57:57 dist-test-slave-h6n0 krb5kdc[8508](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474677, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.112.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:57:57.019623 8697 init.cc:377] Logged in from keytab as oryx/127.29.112.129@KRBTEST.COM (short username oryx)
I20260811 18:57:57.021062 8697 webserver.cc:460] Webserver started at http://127.29.112.129:37475/ using document root <none> and password file <none>
I20260811 18:57:57.021770 8697 fs_manager.cc:359] Metadata directory not provided
I20260811 18:57:57.021914 8697 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:57:57.022220 8697 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:57:57.025168 8697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/ts-0/data/instance:
uuid: "fc3b99856afe4cc3a86cc67240de20e7"
format_stamp: "Formatted at 2026-08-11 18:57:57 on dist-test-slave-h6n0"
server_key: "9b3521c08e4a877c256cd1ac835ef5ec"
server_key_iv: "f1f480d8f08377da78e668152a0caf33"
server_key_version: "encryptionkey@0"
I20260811 18:57:57.025997 8697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/ts-0/wal/instance:
uuid: "fc3b99856afe4cc3a86cc67240de20e7"
format_stamp: "Formatted at 2026-08-11 18:57:57 on dist-test-slave-h6n0"
server_key: "9b3521c08e4a877c256cd1ac835ef5ec"
server_key_iv: "f1f480d8f08377da78e668152a0caf33"
server_key_version: "encryptionkey@0"
I20260811 18:57:57.031533 8697 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 18:57:57.035599 8734 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:57.037813 8697 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 18:57:57.038095 8697 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/ts-0/wal
uuid: "fc3b99856afe4cc3a86cc67240de20e7"
format_stamp: "Formatted at 2026-08-11 18:57:57 on dist-test-slave-h6n0"
server_key: "9b3521c08e4a877c256cd1ac835ef5ec"
server_key_iv: "f1f480d8f08377da78e668152a0caf33"
server_key_version: "encryptionkey@0"
I20260811 18:57:57.038383 8697 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:57:57.058171 8697 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:57:57.062973 8697 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:57:57.063293 8697 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:57:57.064527 8697 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:57:57.066138 8697 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:57:57.066249 8697 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:57.066424 8697 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:57:57.066501 8697 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:57.107853 8697 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.129:35525
I20260811 18:57:57.107951 8939 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.129:35525 every 8 connection(s)
I20260811 18:57:57.109206 8697 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/ts-0/data/info.pb
I20260811 18:57:57.112294 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 8697
I20260811 18:57:57.112541 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/ts-0/wal/instance
I20260811 18:57:57.113196 30146 external_mini_cluster.cc:1428] Setting key b11f0beaa460ad560f46fb86a974dfc6
Aug 11 18:57:57 dist-test-slave-h6n0 krb5kdc[8508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474677, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.112.129@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM
I20260811 18:57:57.133292 8949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:57.112238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.129:49511 (local address 127.29.112.190:37199)
0811 18:57:57.114423 (+ 2185us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:57.114429 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:57.114452 (+ 23us) server_negotiation.cc:407] Connection header received
0811 18:57:57.116506 (+ 2054us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:57.116512 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:57.116601 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:57.116719 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:57.118194 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:57.118984 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:57.120573 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:57.120972 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:57.121105 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:57.125649 (+ 4544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:57.125655 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:57.125658 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:57.125691 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:57.128968 (+ 3277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:57.130199 (+ 1231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:57.130204 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:57.130206 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:57.130363 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:57.131048 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:57.131052 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:57.131054 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:57.131345 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:57.131472 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:57.132664 (+ 1192us) server_negotiation.cc:299] Negotiation successful
0811 18:57:57.132891 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2048,"thread_start_us":1936,"threads_started":1}
I20260811 18:57:57.135152 8952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:57.112961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37199 (local address 127.29.112.129:49511)
0811 18:57:57.114036 (+ 1075us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:57.114089 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:57.116246 (+ 2157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:57.116964 (+ 718us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:57.116972 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:57.117469 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:57:57.118028 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:57.118046 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:57.119150 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:57.119158 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:57.120400 (+ 1242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:57.120412 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:57.121218 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:57.121225 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:57.121746 (+ 521us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:57.122888 (+ 1142us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:57:57.122923 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:57:57.125405 (+ 2482us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:57:57.129173 (+ 3768us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:57.129188 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:57.129211 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:57.130041 (+ 830us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:57.130501 (+ 460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:57.130510 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:57.130518 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:57.130893 (+ 375us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:57.131502 (+ 609us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:57:57.131514 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:57:57.132105 (+ 591us) client_negotiation.cc:769] Sending connection context
0811 18:57:57.132528 (+ 423us) client_negotiation.cc:240] Negotiation successful
0811 18:57:57.133230 (+ 702us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":713,"thread_start_us":442,"threads_started":1}
I20260811 18:57:57.137404 8942 heartbeater.cc:347] Connected to a master server at 127.29.112.190:37199
I20260811 18:57:57.137786 8942 heartbeater.cc:464] Registering TS with master...
I20260811 18:57:57.138751 8942 heartbeater.cc:511] Master 127.29.112.190:37199 requested a full tablet report, sending...
I20260811 18:57:57.141388 8599 ts_manager.cc:194] Registered new tserver with Master: fc3b99856afe4cc3a86cc67240de20e7 (127.29.112.129:35525)
I20260811 18:57:57.143669 8599 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.112.129@KRBTEST.COM'} at 127.29.112.129:49511
WARNING: no policy specified for oryx/127.29.112.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.112.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.112.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/krb5kdc/oryx_127.29.112.130.keytab.
Entry for principal oryx/127.29.112.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/krb5kdc/oryx_127.29.112.130.keytab.
I20260811 18:57:57.182360 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.130:0
--local_ip_for_outbound_sockets=127.29.112.130
--webserver_interface=127.29.112.130
--webserver_port=0
--tserver_master_addrs=127.29.112.190:37199
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/krb5kdc/oryx_127.29.112.130.keytab
--principal=oryx/127.29.112.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:43681
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:57:57.532801 8961 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:57:57.533097 8961 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:57:57.533195 8961 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:57:57.540445 8961 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:57:57.540550 8961 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:57:57.540658 8961 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.130
I20260811 18:57:57.551334 8961 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:43681
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/krb5kdc/oryx_127.29.112.130.keytab
--principal=oryx/127.29.112.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.112.130
--webserver_port=0
--tserver_master_addrs=127.29.112.190:37199
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.130
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:57:57.552641 8961 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:57:57.553920 8961 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:57:57.568890 8975 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:57.568890 8978 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:57.568997 8972 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:57:57.570026 8961 server_base.cc:1034] running on GCE node
I20260811 18:57:57.570880 8961 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:57:57.571941 8961 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:57:57.573179 8961 hybrid_clock.cc:648] HybridClock initialized: now 1786474677573114 us; error 61 us; skew 500 ppm
I20260811 18:57:57.573544 8961 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:57:57 dist-test-slave-h6n0 krb5kdc[8508](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474677, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.112.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:57:57.577090 8961 init.cc:377] Logged in from keytab as oryx/127.29.112.130@KRBTEST.COM (short username oryx)
I20260811 18:57:57.578799 8961 webserver.cc:460] Webserver started at http://127.29.112.130:36757/ using document root <none> and password file <none>
I20260811 18:57:57.579510 8961 fs_manager.cc:359] Metadata directory not provided
I20260811 18:57:57.579617 8961 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:57:57.579921 8961 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:57:57.582999 8961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/ts-1/data/instance:
uuid: "aaab81ed038149258391f548b07268af"
format_stamp: "Formatted at 2026-08-11 18:57:57 on dist-test-slave-h6n0"
server_key: "da63a013e79d4a56ee27953c95881e92"
server_key_iv: "411c0352d123579b1fe7843627aaf21a"
server_key_version: "encryptionkey@0"
I20260811 18:57:57.583819 8961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/ts-1/wal/instance:
uuid: "aaab81ed038149258391f548b07268af"
format_stamp: "Formatted at 2026-08-11 18:57:57 on dist-test-slave-h6n0"
server_key: "da63a013e79d4a56ee27953c95881e92"
server_key_iv: "411c0352d123579b1fe7843627aaf21a"
server_key_version: "encryptionkey@0"
I20260811 18:57:57.589226 8961 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260811 18:57:57.593163 8989 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:57.594847 8961 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 18:57:57.595050 8961 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/ts-1/wal
uuid: "aaab81ed038149258391f548b07268af"
format_stamp: "Formatted at 2026-08-11 18:57:57 on dist-test-slave-h6n0"
server_key: "da63a013e79d4a56ee27953c95881e92"
server_key_iv: "411c0352d123579b1fe7843627aaf21a"
server_key_version: "encryptionkey@0"
I20260811 18:57:57.595253 8961 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:57:57.628705 8961 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:57:57.633100 8961 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:57:57.633409 8961 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:57:57.634368 8961 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:57:57.635761 8961 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:57:57.635839 8961 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:57.635931 8961 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:57:57.636005 8961 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:57.675164 8961 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.130:37583
I20260811 18:57:57.675271 9204 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.130:37583 every 8 connection(s)
I20260811 18:57:57.676481 8961 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/ts-1/data/info.pb
I20260811 18:57:57.679685 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 8961
I20260811 18:57:57.679896 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/ts-1/wal/instance
I20260811 18:57:57.680532 30146 external_mini_cluster.cc:1428] Setting key f0498a39cdb7607cc40dbf16bfa234b8
Aug 11 18:57:57 dist-test-slave-h6n0 krb5kdc[8508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474677, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.112.130@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM
I20260811 18:57:57.703377 9207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:57.679286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.130:47771 (local address 127.29.112.190:37199)
0811 18:57:57.679827 (+ 541us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:57.679834 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:57.680894 (+ 1060us) server_negotiation.cc:407] Connection header received
0811 18:57:57.683217 (+ 2323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:57.683221 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:57.683285 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:57.683381 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:57.686019 (+ 2638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:57.686823 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:57.689349 (+ 2526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:57.689887 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:57.689994 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:57.694755 (+ 4761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:57.694761 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:57.694764 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:57.694804 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:57.698626 (+ 3822us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:57.700125 (+ 1499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:57.700134 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:57.700141 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:57.700314 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:57.700995 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:57.701003 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:57.701010 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:57.701480 (+ 470us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:57.701661 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:57.702494 (+ 833us) server_negotiation.cc:299] Negotiation successful
0811 18:57:57.702845 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":250,"threads_started":1}
I20260811 18:57:57.704151 9209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:57.679854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37199 (local address 127.29.112.130:47771)
0811 18:57:57.680710 (+ 856us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:57.680758 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:57.682909 (+ 2151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:57.683655 (+ 746us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:57.683669 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:57.684463 (+ 794us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:57:57.685685 (+ 1222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:57.685832 (+ 147us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:57.686992 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:57.687000 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:57.689175 (+ 2175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:57.689194 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:57.690139 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:57.690149 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:57.690479 (+ 330us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:57.691304 (+ 825us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:57:57.691346 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:57:57.694420 (+ 3074us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:57:57.698983 (+ 4563us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:57.699001 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:57.699029 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:57.699779 (+ 750us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:57.700509 (+ 730us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:57.700514 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:57.700517 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:57.700845 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:57.701676 (+ 831us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:57:57.701684 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:57:57.702100 (+ 416us) client_negotiation.cc:769] Sending connection context
0811 18:57:57.702374 (+ 274us) client_negotiation.cc:240] Negotiation successful
0811 18:57:57.702924 (+ 550us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":578,"thread_start_us":342,"threads_started":1}
I20260811 18:57:57.706292 9205 heartbeater.cc:347] Connected to a master server at 127.29.112.190:37199
I20260811 18:57:57.706693 9205 heartbeater.cc:464] Registering TS with master...
I20260811 18:57:57.707480 9205 heartbeater.cc:511] Master 127.29.112.190:37199 requested a full tablet report, sending...
I20260811 18:57:57.709118 8599 ts_manager.cc:194] Registered new tserver with Master: aaab81ed038149258391f548b07268af (127.29.112.130:37583)
I20260811 18:57:57.709944 8599 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.112.130@KRBTEST.COM'} at 127.29.112.130:47771
WARNING: no policy specified for oryx/127.29.112.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.112.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.112.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/krb5kdc/oryx_127.29.112.131.keytab.
Entry for principal oryx/127.29.112.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/krb5kdc/oryx_127.29.112.131.keytab.
I20260811 18:57:57.747165 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.131:0
--local_ip_for_outbound_sockets=127.29.112.131
--webserver_interface=127.29.112.131
--webserver_port=0
--tserver_master_addrs=127.29.112.190:37199
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/krb5kdc/oryx_127.29.112.131.keytab
--principal=oryx/127.29.112.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:43681
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:57:58.085752 9216 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:57:58.086025 9216 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:57:58.086120 9216 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:57:58.094310 9216 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:57:58.094429 9216 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:57:58.094622 9216 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.131
I20260811 18:57:58.104511 9216 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:43681
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/krb5kdc/oryx_127.29.112.131.keytab
--principal=oryx/127.29.112.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.112.131
--webserver_port=0
--tserver_master_addrs=127.29.112.190:37199
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.131
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:57:58.105737 9216 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:57:58.107079 9216 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:57:58.119994 9221 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:58.120052 9222 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:58.120092 9226 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:57:58.120848 9216 server_base.cc:1034] running on GCE node
I20260811 18:57:58.121381 9216 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:57:58.122282 9216 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:57:58.123517 9216 hybrid_clock.cc:648] HybridClock initialized: now 1786474678123432 us; error 58 us; skew 500 ppm
I20260811 18:57:58.123940 9216 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:57:58 dist-test-slave-h6n0 krb5kdc[8508](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474678, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.112.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:57:58.128461 9216 init.cc:377] Logged in from keytab as oryx/127.29.112.131@KRBTEST.COM (short username oryx)
I20260811 18:57:58.130167 9216 webserver.cc:460] Webserver started at http://127.29.112.131:33049/ using document root <none> and password file <none>
I20260811 18:57:58.131039 9216 fs_manager.cc:359] Metadata directory not provided
I20260811 18:57:58.131198 9216 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:57:58.131510 9216 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:57:58.134392 9216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/ts-2/data/instance:
uuid: "fa32c49cd161420c979c48cd9fad3b05"
format_stamp: "Formatted at 2026-08-11 18:57:58 on dist-test-slave-h6n0"
server_key: "9a678c2cb558f7b88b9c11b75f99ef21"
server_key_iv: "47a019c22dcd7bd0696ca97b80f1ecb3"
server_key_version: "encryptionkey@0"
I20260811 18:57:58.135272 9216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/ts-2/wal/instance:
uuid: "fa32c49cd161420c979c48cd9fad3b05"
format_stamp: "Formatted at 2026-08-11 18:57:58 on dist-test-slave-h6n0"
server_key: "9a678c2cb558f7b88b9c11b75f99ef21"
server_key_iv: "47a019c22dcd7bd0696ca97b80f1ecb3"
server_key_version: "encryptionkey@0"
I20260811 18:57:58.141115 9216 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260811 18:57:58.145754 9237 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:58.147168 9216 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:57:58.147351 9216 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/ts-2/wal
uuid: "fa32c49cd161420c979c48cd9fad3b05"
format_stamp: "Formatted at 2026-08-11 18:57:58 on dist-test-slave-h6n0"
server_key: "9a678c2cb558f7b88b9c11b75f99ef21"
server_key_iv: "47a019c22dcd7bd0696ca97b80f1ecb3"
server_key_version: "encryptionkey@0"
I20260811 18:57:58.147539 9216 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:57:58.147922 8942 heartbeater.cc:503] Master 127.29.112.190:37199 was elected leader, sending a full tablet report...
I20260811 18:57:58.187122 9216 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:57:58.192991 9216 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:57:58.193349 9216 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:57:58.194394 9216 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:57:58.195945 9216 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:57:58.196034 9216 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:58.196185 9216 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:57:58.196244 9216 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:58.239580 9216 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.131:41101
I20260811 18:57:58.239706 9444 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.131:41101 every 8 connection(s)
I20260811 18:57:58.241066 9216 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/ts-2/data/info.pb
I20260811 18:57:58.249260 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 9216
I20260811 18:57:58.249491 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/ts-2/wal/instance
I20260811 18:57:58.250034 30146 external_mini_cluster.cc:1428] Setting key b04da6069f72dd92a1b63b9d75b3c50b
Aug 11 18:57:58 dist-test-slave-h6n0 krb5kdc[8508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474678, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.112.131@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM
I20260811 18:57:58.267318 9449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:58.244557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.131:47921 (local address 127.29.112.190:37199)
0811 18:57:58.245495 (+ 938us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:58.245500 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:58.246371 (+ 871us) server_negotiation.cc:407] Connection header received
0811 18:57:58.248227 (+ 1856us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:58.248235 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:58.248333 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:58.248460 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:58.250626 (+ 2166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:58.251932 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:58.254037 (+ 2105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:58.254487 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:58.254708 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:58.259332 (+ 4624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:58.259338 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:58.259341 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:58.259377 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:58.262864 (+ 3487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:58.263945 (+ 1081us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:58.263951 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:58.263957 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:58.264123 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:58.264913 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:58.264920 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:58.264924 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:58.265382 (+ 458us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:58.265576 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:58.266646 (+ 1070us) server_negotiation.cc:299] Negotiation successful
0811 18:57:58.266941 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":835,"thread_start_us":701,"threads_started":1}
I20260811 18:57:58.268366 9450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:58.245429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37199 (local address 127.29.112.131:47921)
0811 18:57:58.246186 (+ 757us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:58.246224 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:58.247938 (+ 1714us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:58.248698 (+ 760us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:58.248706 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:58.249522 (+ 816us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:57:58.250362 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:58.250398 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:58.252264 (+ 1866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:58.252272 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:58.253877 (+ 1605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:58.253893 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:58.254807 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:58.254815 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:58.255304 (+ 489us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:58.256455 (+ 1151us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:57:58.256481 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:57:58.259083 (+ 2602us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:57:58.263131 (+ 4048us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:58.263147 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:58.263168 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:58.263792 (+ 624us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:58.264426 (+ 634us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:58.264433 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:58.264437 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:58.264764 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:58.265540 (+ 776us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:57:58.265583 (+ 43us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:57:58.266103 (+ 520us) client_negotiation.cc:769] Sending connection context
0811 18:57:58.266468 (+ 365us) client_negotiation.cc:240] Negotiation successful
0811 18:57:58.267124 (+ 656us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":307,"threads_started":1}
I20260811 18:57:58.270088 9445 heartbeater.cc:347] Connected to a master server at 127.29.112.190:37199
I20260811 18:57:58.270881 9445 heartbeater.cc:464] Registering TS with master...
I20260811 18:57:58.273474 9445 heartbeater.cc:511] Master 127.29.112.190:37199 requested a full tablet report, sending...
I20260811 18:57:58.276511 8599 ts_manager.cc:194] Registered new tserver with Master: fa32c49cd161420c979c48cd9fad3b05 (127.29.112.131:41101)
I20260811 18:57:58.278250 8599 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.112.131@KRBTEST.COM'} at 127.29.112.131:47921
I20260811 18:57:58.281104 30146 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.29.112.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.112.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.112.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/krb5kdc/oryx_127.29.112.189.keytab.
Entry for principal oryx/127.29.112.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/krb5kdc/oryx_127.29.112.189.keytab.
I20260811 18:57:58.350244 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.189:45725
--webserver_interface=127.29.112.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/krb5kdc/oryx_127.29.112.189.keytab
--principal=oryx/127.29.112.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:43681
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.190:37199,127.29.112.189:45725
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:57:58.704032 9454 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:57:58.704367 9454 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:57:58.704542 9454 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:57:58.711651 9454 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:57:58.711797 9454 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:57:58.711911 9454 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:57:58.712025 9454 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:57:58.712119 9454 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:57:58.712572 9205 heartbeater.cc:503] Master 127.29.112.190:37199 was elected leader, sending a full tablet report...
I20260811 18:57:58.722260 9454 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:43681
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.29.112.190:37199,127.29.112.189:45725
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/krb5kdc/oryx_127.29.112.189.keytab
--principal=oryx/127.29.112.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.189:45725
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.29.112.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
W20260811 18:57:58.722695 9454 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.29.112.190:37199,127.29.112.189:45725'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260811 18:57:58.723758 9454 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:57:58.725059 9454 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:57:58.736941 9461 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:58.736974 9465 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:58.737016 9460 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:57:58.741545 9454 server_base.cc:1034] running on GCE node
I20260811 18:57:58.742449 9454 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:57:58.743934 9454 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:57:58.745174 9454 hybrid_clock.cc:648] HybridClock initialized: now 1786474678745098 us; error 45 us; skew 500 ppm
I20260811 18:57:58.745543 9454 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:57:58 dist-test-slave-h6n0 krb5kdc[8508](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474678, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.112.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:57:58.749030 9454 init.cc:377] Logged in from keytab as oryx/127.29.112.189@KRBTEST.COM (short username oryx)
I20260811 18:57:58.750648 9454 webserver.cc:460] Webserver started at http://127.29.112.189:46875/ using document root <none> and password file <none>
I20260811 18:57:58.751422 9454 fs_manager.cc:359] Metadata directory not provided
I20260811 18:57:58.751536 9454 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:57:58.751886 9454 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:57:58.754904 9454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/master-1/data/instance:
uuid: "ea739164f1b344099a538c6c91d0ada6"
format_stamp: "Formatted at 2026-08-11 18:57:58 on dist-test-slave-h6n0"
server_key: "4bc2f005276964e4f5e3e417f1f2d29e"
server_key_iv: "098e6a5e33ff0279ac95c461578a99a0"
server_key_version: "encryptionkey@0"
I20260811 18:57:58.755709 9454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/master-1/wal/instance:
uuid: "ea739164f1b344099a538c6c91d0ada6"
format_stamp: "Formatted at 2026-08-11 18:57:58 on dist-test-slave-h6n0"
server_key: "4bc2f005276964e4f5e3e417f1f2d29e"
server_key_iv: "098e6a5e33ff0279ac95c461578a99a0"
server_key_version: "encryptionkey@0"
I20260811 18:57:58.761466 9454 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 18:57:58.766076 9482 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:58.767712 9454 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 18:57:58.767920 9454 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/master-1/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/master-1/wal
uuid: "ea739164f1b344099a538c6c91d0ada6"
format_stamp: "Formatted at 2026-08-11 18:57:58 on dist-test-slave-h6n0"
server_key: "4bc2f005276964e4f5e3e417f1f2d29e"
server_key_iv: "098e6a5e33ff0279ac95c461578a99a0"
server_key_version: "encryptionkey@0"
I20260811 18:57:58.768213 9454 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:57:58.793431 9454 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:57:58.797700 9454 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:57:58.797940 9454 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:57:58.819829 9454 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.189:45725
I20260811 18:57:58.819854 9563 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.189:45725 every 8 connection(s)
I20260811 18:57:58.821682 9454 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/master-1/data/info.pb
I20260811 18:57:58.825594 9565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:58.832473 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 9454
I20260811 18:57:58.833734 9565 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.112.190" port: 37199 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 18:57:58 dist-test-slave-h6n0 krb5kdc[8508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474678, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.112.189@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM
Aug 11 18:57:58 dist-test-slave-h6n0 krb5kdc[8508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474675, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.29.112.189@KRBTEST.COM
I20260811 18:57:58.859923 9576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:58.837009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40516 (local address 127.29.112.190:37199)
0811 18:57:58.839044 (+ 2035us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:58.839050 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:58.839077 (+ 27us) server_negotiation.cc:407] Connection header received
0811 18:57:58.839910 (+ 833us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:58.839916 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:58.840025 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:58.840316 (+ 291us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:58.842176 (+ 1860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:58.843610 (+ 1434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:58.845894 (+ 2284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:58.846692 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:58.846843 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:58.851903 (+ 5060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:58.851910 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:58.851913 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:58.851961 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:58.855759 (+ 3798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:58.856692 (+ 933us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:58.856699 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:58.856706 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:58.856851 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:58.857684 (+ 833us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:58.857688 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:58.857690 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:58.858212 (+ 522us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:58.858323 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:58.859243 (+ 920us) server_negotiation.cc:299] Negotiation successful
0811 18:57:58.859527 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1883,"thread_start_us":1752,"threads_started":1}
I20260811 18:57:58.860944 9573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:58.835367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51660 (local address 127.29.112.189:45725)
0811 18:57:58.836248 (+ 881us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:58.836263 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:58.836311 (+ 48us) server_negotiation.cc:407] Connection header received
0811 18:57:58.839377 (+ 3066us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:58.839404 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:58.839711 (+ 307us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:58.839971 (+ 260us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:58.842130 (+ 2159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:58.844282 (+ 2152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:58.846036 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:58.847075 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:58.847248 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:58.852660 (+ 5412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:58.852672 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:58.852686 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:58.852717 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:58.856604 (+ 3887us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:58.857845 (+ 1241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:58.857853 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:58.857856 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:58.858011 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:58.858490 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:58.858494 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:58.858496 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:58.858924 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:58.859211 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:58.859605 (+ 394us) server_negotiation.cc:299] Negotiation successful
0811 18:57:58.859859 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":64,"server-negotiator.queue_time_us":654,"thread_start_us":447,"threads_started":1}
I20260811 18:57:58.860987 9575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:58.836884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37199 (local address 127.0.0.1:40516)
0811 18:57:58.838308 (+ 1424us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:58.838396 (+ 88us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:58.839415 (+ 1019us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:58.840392 (+ 977us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:58.840401 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:58.841366 (+ 965us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:57:58.841971 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:58.842000 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:58.843787 (+ 1787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:58.843791 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:58.845692 (+ 1901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:58.845708 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:58.846973 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:58.846978 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:58.847255 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:58.848286 (+ 1031us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:57:58.848318 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:57:58.851646 (+ 3328us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:57:58.855938 (+ 4292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:58.855956 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:58.855970 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:58.856566 (+ 596us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:58.857107 (+ 541us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:58.857114 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:58.857119 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:58.857527 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:58.858375 (+ 848us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:57:58.858384 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:57:58.858819 (+ 435us) client_negotiation.cc:769] Sending connection context
0811 18:57:58.859134 (+ 315us) client_negotiation.cc:240] Negotiation successful
0811 18:57:58.859669 (+ 535us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":971,"mutex_wait_us":5,"thread_start_us":557,"threads_started":1}
I20260811 18:57:58.862941 9565 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.112.189" port: 45725 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 18:57:58 dist-test-slave-h6n0 krb5kdc[8508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474678, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.112.189@KRBTEST.COM for oryx/127.29.112.189@KRBTEST.COM
I20260811 18:57:58.879133 9575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:58.863787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.189:45725 (local address 127.0.0.1:51672)
0811 18:57:58.864062 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:58.864082 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:58.864189 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:58.864479 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:58.864483 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:58.864831 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:57:58.864959 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:58.864967 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:58.865930 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:58.865935 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:58.866864 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:58.866875 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:58.867518 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:58.867523 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:58.867749 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:58.868326 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:57:58.868344 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:57:58.870633 (+ 2289us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:57:58.875556 (+ 4923us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:58.875561 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:58.875568 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:58.876180 (+ 612us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:58.876795 (+ 615us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:58.876802 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:58.876808 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:58.876976 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:58.877835 (+ 859us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:57:58.877840 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:57:58.877994 (+ 154us) client_negotiation.cc:769] Sending connection context
0811 18:57:58.878155 (+ 161us) client_negotiation.cc:240] Negotiation successful
0811 18:57:58.878708 (+ 553us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"mutex_wait_us":23}
I20260811 18:57:58.879184 9573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:58.863888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51672 (local address 127.29.112.189:45725)
0811 18:57:58.864063 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:58.864069 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:58.864151 (+ 82us) server_negotiation.cc:407] Connection header received
0811 18:57:58.864301 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:58.864306 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:58.864370 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:58.864465 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:58.865086 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:58.865786 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:58.866997 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:58.867379 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:58.867518 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:58.871038 (+ 3520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:58.871046 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:58.871050 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:58.871086 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:58.875352 (+ 4266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:58.876483 (+ 1131us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:58.876490 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:58.876494 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:58.876668 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:58.877086 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:58.877093 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:58.877097 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:58.877662 (+ 565us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:58.877882 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:58.878457 (+ 575us) server_negotiation.cc:299] Negotiation successful
0811 18:57:58.878896 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":146,"server-negotiator.queue_time_us":57}
I20260811 18:57:58.885280 9565 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea739164f1b344099a538c6c91d0ada6: Bootstrap starting.
I20260811 18:57:58.889389 9565 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ea739164f1b344099a538c6c91d0ada6: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:58.890713 9565 log.cc:824] T 00000000000000000000000000000000 P ea739164f1b344099a538c6c91d0ada6: Log is configured to *not* fsync() on all Append() calls
I20260811 18:57:58.894340 9565 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea739164f1b344099a538c6c91d0ada6: No bootstrap required, opened a new log
I20260811 18:57:58.902905 9565 raft_consensus.cc:348] T 00000000000000000000000000000000 P ea739164f1b344099a538c6c91d0ada6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b994bc87dbdf4d93a9b50500ad3f2aca" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 37199 } } peers { permanent_uuid: "ea739164f1b344099a538c6c91d0ada6" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 45725 } }
I20260811 18:57:58.903506 9565 raft_consensus.cc:374] T 00000000000000000000000000000000 P ea739164f1b344099a538c6c91d0ada6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:58.903646 9565 raft_consensus.cc:729] T 00000000000000000000000000000000 P ea739164f1b344099a538c6c91d0ada6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea739164f1b344099a538c6c91d0ada6, State: Initialized, Role: FOLLOWER
I20260811 18:57:58.904312 9565 consensus_queue.cc:260] T 00000000000000000000000000000000 P ea739164f1b344099a538c6c91d0ada6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b994bc87dbdf4d93a9b50500ad3f2aca" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 37199 } } peers { permanent_uuid: "ea739164f1b344099a538c6c91d0ada6" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 45725 } }
I20260811 18:57:58.905216 9579 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea739164f1b344099a538c6c91d0ada6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b994bc87dbdf4d93a9b50500ad3f2aca" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 37199 } } peers { permanent_uuid: "ea739164f1b344099a538c6c91d0ada6" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 45725 } } }
I20260811 18:57:58.905544 9579 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea739164f1b344099a538c6c91d0ada6 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:57:58.906270 9565 sys_catalog.cc:564] T 00000000000000000000000000000000 P ea739164f1b344099a538c6c91d0ada6 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 18:57:58.915797 9596 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ea739164f1b344099a538c6c91d0ada6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:57:58.916071 9596 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:57:58.920233 9454 master_runner.cc:428] Detected that this master ea739164f1b344099a538c6c91d0ada6 is joining an existing cluster
I20260811 18:57:58.920341 9454 master_runner.cc:432] Initiating AddMaster RPC to add 127.29.112.189:45725
I20260811 18:57:58.944651 9573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:58.925060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51674 (local address 127.29.112.189:45725)
0811 18:57:58.925296 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:58.925304 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:58.927265 (+ 1961us) server_negotiation.cc:407] Connection header received
0811 18:57:58.927458 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:58.927465 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:58.927555 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:58.927659 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:58.929721 (+ 2062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:58.930970 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:58.933846 (+ 2876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:58.934653 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:58.934753 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:58.937363 (+ 2610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:58.937370 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:58.937376 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:58.937414 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:58.941377 (+ 3963us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:58.942276 (+ 899us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:58.942281 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:58.942286 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:58.942469 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:58.942884 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:58.942889 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:58.942892 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:58.943328 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:58.943481 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:58.943987 (+ 506us) server_negotiation.cc:299] Negotiation successful
0811 18:57:58.944226 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 18:57:58.945703 9576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:58.924396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40524 (local address 127.29.112.190:37199)
0811 18:57:58.924612 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:58.924621 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:58.930019 (+ 5398us) server_negotiation.cc:407] Connection header received
0811 18:57:58.930210 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:58.930216 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:58.930305 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:58.930401 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:58.931863 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:58.932966 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:58.934990 (+ 2024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:58.935646 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:58.935717 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:58.938287 (+ 2570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:58.938319 (+ 32us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:58.938325 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:58.938378 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:58.942363 (+ 3985us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:58.943129 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:58.943136 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:58.943141 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:58.943348 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:58.943784 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:58.943789 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:58.943792 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:58.944132 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:58.944269 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:58.945116 (+ 847us) server_negotiation.cc:299] Negotiation successful
0811 18:57:58.945360 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 18:57:58.945748 9603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:58.924632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.189:45725 (local address 127.0.0.1:51674)
0811 18:57:58.927176 (+ 2544us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:58.927195 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:58.927331 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:58.927869 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:58.927876 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:58.929001 (+ 1125us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:57:58.929522 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:58.929539 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:58.932152 (+ 2613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:58.932158 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:58.933711 (+ 1553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:58.933725 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:58.935069 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:58.935075 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:58.935262 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:58.935954 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:57:58.935975 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:57:58.937142 (+ 1167us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:57:58.941518 (+ 4376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:58.941523 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:58.941527 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:58.942158 (+ 631us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:58.942619 (+ 461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:58.942623 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:58.942626 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:58.942758 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:58.943563 (+ 805us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:57:58.943570 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:57:58.943710 (+ 140us) client_negotiation.cc:769] Sending connection context
0811 18:57:58.944976 (+ 1266us) client_negotiation.cc:240] Negotiation successful
0811 18:57:58.945276 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2415,"thread_start_us":2264,"threads_started":1}
WARNING: no policy specified for oryx/127.29.112.188@KRBTEST.COM; defaulting to no policy
I20260811 18:57:58.947572 9604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:58.924636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37199 (local address 127.0.0.1:40524)
0811 18:57:58.929914 (+ 5278us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:58.929934 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:58.930062 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:58.930603 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:58.930608 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:58.931298 (+ 690us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:57:58.931686 (+ 388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:58.931698 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:58.933159 (+ 1461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:58.933164 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:58.934804 (+ 1640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:58.934819 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:58.936046 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:58.936051 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:58.936240 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:58.936818 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:57:58.936854 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:57:58.938050 (+ 1196us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:57:58.942504 (+ 4454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:58.942510 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:58.942516 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:58.942983 (+ 467us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:58.943484 (+ 501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:58.943493 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:58.943498 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:58.943657 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:58.944363 (+ 706us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:57:58.944369 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:57:58.944575 (+ 206us) client_negotiation.cc:769] Sending connection context
0811 18:57:58.946689 (+ 2114us) client_negotiation.cc:240] Negotiation successful
0811 18:57:58.947018 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5113,"mutex_wait_us":24,"thread_start_us":4896,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.112.188@KRBTEST.COM" created.
Aug 11 18:57:58 dist-test-slave-h6n0 krb5kdc[8508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474676, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.112.190@KRBTEST.COM for oryx/127.29.112.189@KRBTEST.COM
I20260811 18:57:58.968523 9573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:58.954032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51680 (local address 127.29.112.189:45725)
0811 18:57:58.954199 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:58.954205 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:58.954514 (+ 309us) server_negotiation.cc:407] Connection header received
0811 18:57:58.954913 (+ 399us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:58.954918 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:58.955022 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:58.955135 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:58.956175 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:58.956869 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:58.958196 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:58.958774 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:58.958916 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:58.962720 (+ 3804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:58.962726 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:58.962730 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:58.962766 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:58.965789 (+ 3023us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:58.966478 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:58.966487 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:58.966490 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:58.966662 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:58.967055 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:58.967060 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:58.967064 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:58.967304 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:58.967467 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:58.967921 (+ 454us) server_negotiation.cc:299] Negotiation successful
0811 18:57:58.968145 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 18:57:58.968575 9608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:58.953779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.189:45725 (local address 127.0.0.1:51680)
0811 18:57:58.954389 (+ 610us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:58.954435 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:58.954685 (+ 250us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:58.955182 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:58.955189 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:58.955796 (+ 607us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:57:58.956015 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:58.956024 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:58.957010 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:58.957015 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:58.958056 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:58.958068 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:58.958931 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:58.958937 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:58.959133 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:58.959824 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:57:58.959852 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:57:58.962474 (+ 2622us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:57:58.965926 (+ 3452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:58.965939 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:58.965946 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:58.966339 (+ 393us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:58.966777 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:58.966782 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:58.966785 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:58.966930 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:58.967436 (+ 506us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:57:58.967444 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:57:58.967630 (+ 186us) client_negotiation.cc:769] Sending connection context
0811 18:57:58.967818 (+ 188us) client_negotiation.cc:240] Negotiation successful
0811 18:57:58.968140 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":443,"threads_started":1}
I20260811 18:57:58.971027 8599 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.29.112.189:45725: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "ea739164f1b344099a538c6c91d0ada6"
member_type: NON_VOTER
last_known_addr {
host: "127.29.112.189"
port: 45725
}
attrs {
promote: true
}
}
dest_uuid: "b994bc87dbdf4d93a9b50500ad3f2aca"
cas_config_opid_index: -1
I20260811 18:57:58.972256 8599 consensus_queue.cc:237] T 00000000000000000000000000000000 P b994bc87dbdf4d93a9b50500ad3f2aca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b994bc87dbdf4d93a9b50500ad3f2aca" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 37199 } } peers { permanent_uuid: "ea739164f1b344099a538c6c91d0ada6" member_type: NON_VOTER last_known_addr { host: "127.29.112.189" port: 45725 } attrs { promote: true } }
I20260811 18:57:58.980659 9524 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ea739164f1b344099a538c6c91d0ada6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:58.979990 9613 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b994bc87dbdf4d93a9b50500ad3f2aca [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER ea739164f1b344099a538c6c91d0ada6 (127.29.112.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b994bc87dbdf4d93a9b50500ad3f2aca" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 37199 } } peers { permanent_uuid: "ea739164f1b344099a538c6c91d0ada6" member_type: NON_VOTER last_known_addr { host: "127.29.112.189" port: 45725 } attrs { promote: true } } }
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.112.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/krb5kdc/oryx_127.29.112.188.keytab.
Entry for principal oryx/127.29.112.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/krb5kdc/oryx_127.29.112.188.keytab.
I20260811 18:57:58.984851 9524 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ea739164f1b344099a538c6c91d0ada6 [term 1 FOLLOWER]: Refusing update from remote peer b994bc87dbdf4d93a9b50500ad3f2aca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260811 18:57:58.984881 9613 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.29.112.189:45725
I20260811 18:57:58.986344 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.188:44651
--webserver_interface=127.29.112.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/krb5kdc/oryx_127.29.112.188.keytab
--principal=oryx/127.29.112.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:43681
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.190:37199,127.29.112.189:45725,127.29.112.188:44651
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 18:57:58.986892 9454 master.cc:537] Master@127.29.112.189:45725 shutting down...
I20260811 18:57:58.987010 9614 sys_catalog.cc:455] T 00000000000000000000000000000000 P b994bc87dbdf4d93a9b50500ad3f2aca [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "b994bc87dbdf4d93a9b50500ad3f2aca" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b994bc87dbdf4d93a9b50500ad3f2aca" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 37199 } } peers { permanent_uuid: "ea739164f1b344099a538c6c91d0ada6" member_type: NON_VOTER last_known_addr { host: "127.29.112.189" port: 45725 } attrs { promote: true } } }
I20260811 18:57:58.987401 9614 sys_catalog.cc:458] T 00000000000000000000000000000000 P b994bc87dbdf4d93a9b50500ad3f2aca [sys.catalog]: This master's current role is: LEADER
I20260811 18:57:58.986177 9615 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b994bc87dbdf4d93a9b50500ad3f2aca [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea739164f1b344099a538c6c91d0ada6" member_type: NON_VOTER last_known_addr { host: "127.29.112.189" port: 45725 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 18:57:58.991926 8567 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260811 18:57:59.006800 8567 consensus_peers.cc:487] T 00000000000000000000000000000000 P b994bc87dbdf4d93a9b50500ad3f2aca -> Peer ea739164f1b344099a538c6c91d0ada6 (127.29.112.189:45725): Couldn't send request to peer ea739164f1b344099a538c6c91d0ada6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:59.007182 9454 raft_consensus.cc:2229] T 00000000000000000000000000000000 P ea739164f1b344099a538c6c91d0ada6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:59.007640 9454 raft_consensus.cc:2258] T 00000000000000000000000000000000 P ea739164f1b344099a538c6c91d0ada6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:59.007894 9454 tablet_replica.cc:330] T 00000000000000000000000000000000 P ea739164f1b344099a538c6c91d0ada6: stopping tablet replica
I20260811 18:57:59.014672 9454 master.cc:552] Master@127.29.112.189:45725 shutdown complete.
I20260811 18:57:59.014770 9454 master_runner.cc:305] Clearing existing system tablet
I20260811 18:57:59.015204 9454 fs_manager.cc:359] Metadata directory not provided
I20260811 18:57:59.015323 9454 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:57:59.017611 9454 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260811 18:57:59.019349 9624 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:59.019935 9454 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:57:59.020291 9454 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 18:57:59.020452 9454 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/master-1/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/master-1/wal
uuid: "ea739164f1b344099a538c6c91d0ada6"
format_stamp: "Formatted at 2026-08-11 18:57:58 on dist-test-slave-h6n0"
server_key: "4bc2f005276964e4f5e3e417f1f2d29e"
server_key_iv: "098e6a5e33ff0279ac95c461578a99a0"
server_key_version: "encryptionkey@0"
I20260811 18:57:59.024187 9454 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P ea739164f1b344099a538c6c91d0ada6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:59.028931 9454 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P ea739164f1b344099a538c6c91d0ada6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260811 18:57:59.029119 9454 log.cc:1196] T 00000000000000000000000000000000 P ea739164f1b344099a538c6c91d0ada6: Deleting WAL directory at /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260811 18:57:59.029520 9454 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P ea739164f1b344099a538c6c91d0ada6: Deleting consensus metadata
I20260811 18:57:59.029775 9454 master_runner.cc:315] Copying system tablet from 127.29.112.190:37199
I20260811 18:57:59.032270 9454 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P ea739164f1b344099a538c6c91d0ada6: tablet copy: Beginning tablet copy session from remote peer at address 127.29.112.190:37199
I20260811 18:57:59.046820 9576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:59.033060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60510 (local address 127.29.112.190:37199)
0811 18:57:59.033275 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:59.033286 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:59.033852 (+ 566us) server_negotiation.cc:407] Connection header received
0811 18:57:59.034058 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:59.034063 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:59.034159 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:59.034289 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:59.035766 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:59.036591 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:59.037936 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:59.038424 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:59.038590 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:59.040610 (+ 2020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:59.040617 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:59.040624 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:59.040664 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:59.044152 (+ 3488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:59.044774 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:59.044782 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:59.044786 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:59.044923 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:59.045292 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:59.045297 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:59.045301 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:59.045586 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:59.045751 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:59.046168 (+ 417us) server_negotiation.cc:299] Negotiation successful
0811 18:57:59.046380 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 18:57:59.046876 9636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:59.033057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37199 (local address 127.0.0.1:60510)
0811 18:57:59.033730 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:59.033756 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:59.033922 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:59.034314 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:59.034318 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:59.035111 (+ 793us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:57:59.035575 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:59.035587 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:59.036751 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:59.036758 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:59.037762 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:59.037774 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:59.038622 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:59.038629 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:59.038853 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:59.039580 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:57:59.039600 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:57:59.040399 (+ 799us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:57:59.044281 (+ 3882us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:59.044287 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:59.044292 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:59.044655 (+ 363us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:59.045039 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:59.045044 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:59.045048 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:59.045181 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:59.045721 (+ 540us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:57:59.045726 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:57:59.045921 (+ 195us) client_negotiation.cc:769] Sending connection context
0811 18:57:59.046075 (+ 154us) client_negotiation.cc:240] Negotiation successful
0811 18:57:59.046362 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":537,"spinlock_wait_cycles":8064,"thread_start_us":514,"threads_started":1}
I20260811 18:57:59.048026 8624 tablet_copy_service.cc:140] P b994bc87dbdf4d93a9b50500ad3f2aca: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer ea739164f1b344099a538c6c91d0ada6 ({username='oryx', principal='oryx/127.29.112.189@KRBTEST.COM'} at 127.0.0.1:60510)
I20260811 18:57:59.048342 8624 tablet_copy_service.cc:161] P b994bc87dbdf4d93a9b50500ad3f2aca: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer ea739164f1b344099a538c6c91d0ada6 at {username='oryx', principal='oryx/127.29.112.189@KRBTEST.COM'} at 127.0.0.1:60510: session id = ea739164f1b344099a538c6c91d0ada6-00000000000000000000000000000000
I20260811 18:57:59.051636 8624 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P b994bc87dbdf4d93a9b50500ad3f2aca: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 18:57:59.053536 9454 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260811 18:57:59.058990 9454 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P ea739164f1b344099a538c6c91d0ada6: tablet copy: Starting download of 0 data blocks...
I20260811 18:57:59.059302 9454 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P ea739164f1b344099a538c6c91d0ada6: tablet copy: Starting download of 1 WAL segments...
I20260811 18:57:59.061609 9454 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P ea739164f1b344099a538c6c91d0ada6: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 18:57:59.064810 8624 tablet_copy_service.cc:342] P b994bc87dbdf4d93a9b50500ad3f2aca: Request end of tablet copy session ea739164f1b344099a538c6c91d0ada6-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.29.112.189@KRBTEST.COM'} at 127.0.0.1:60510
I20260811 18:57:59.064994 8624 tablet_copy_service.cc:433] P b994bc87dbdf4d93a9b50500ad3f2aca: ending tablet copy session ea739164f1b344099a538c6c91d0ada6-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer ea739164f1b344099a538c6c91d0ada6
I20260811 18:57:59.075577 9454 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:57:59.078135 9637 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:59.084175 9638 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:59.088861 9641 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:57:59.089015 9454 server_base.cc:1034] running on GCE node
I20260811 18:57:59.089476 9454 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:57:59.090169 9454 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:57:59.091413 9454 hybrid_clock.cc:648] HybridClock initialized: now 1786474679091332 us; error 48 us; skew 500 ppm
I20260811 18:57:59.091642 9454 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:57:59 dist-test-slave-h6n0 krb5kdc[8508](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474679, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.112.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:57:59.093528 9454 init.cc:377] Logged in from keytab as oryx/127.29.112.189@KRBTEST.COM (short username oryx)
I20260811 18:57:59.099190 9454 webserver.cc:460] Webserver started at http://127.29.112.189:40425/ using document root <none> and password file <none>
I20260811 18:57:59.099635 9454 fs_manager.cc:359] Metadata directory not provided
I20260811 18:57:59.099745 9454 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:57:59.102398 9454 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260811 18:57:59.104113 9657 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:59.104545 9454 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 18:57:59.104750 9454 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/master-1/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/master-1/wal
uuid: "ea739164f1b344099a538c6c91d0ada6"
format_stamp: "Formatted at 2026-08-11 18:57:58 on dist-test-slave-h6n0"
server_key: "4bc2f005276964e4f5e3e417f1f2d29e"
server_key_iv: "098e6a5e33ff0279ac95c461578a99a0"
server_key_version: "encryptionkey@0"
I20260811 18:57:59.104923 9454 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:57:59.126272 9454 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:57:59.130064 9454 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:57:59.161947 9454 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.189:45725
I20260811 18:57:59.161966 9747 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.189:45725 every 8 connection(s)
I20260811 18:57:59.162981 9454 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/master-1/data/info.pb
I20260811 18:57:59.171136 9748 sys_catalog.cc:263] Verifying existing consensus state
I20260811 18:57:59.175743 9748 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea739164f1b344099a538c6c91d0ada6: Bootstrap starting.
I20260811 18:57:59.190819 9748 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea739164f1b344099a538c6c91d0ada6: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 18:57:59.191368 9748 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea739164f1b344099a538c6c91d0ada6: Bootstrap complete.
I20260811 18:57:59.192261 9748 raft_consensus.cc:348] T 00000000000000000000000000000000 P ea739164f1b344099a538c6c91d0ada6 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b994bc87dbdf4d93a9b50500ad3f2aca" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 37199 } } peers { permanent_uuid: "ea739164f1b344099a538c6c91d0ada6" member_type: NON_VOTER last_known_addr { host: "127.29.112.189" port: 45725 } attrs { promote: true } }
I20260811 18:57:59.192498 9748 raft_consensus.cc:729] T 00000000000000000000000000000000 P ea739164f1b344099a538c6c91d0ada6 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ea739164f1b344099a538c6c91d0ada6, State: Initialized, Role: LEARNER
I20260811 18:57:59.192734 9748 consensus_queue.cc:260] T 00000000000000000000000000000000 P ea739164f1b344099a538c6c91d0ada6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b994bc87dbdf4d93a9b50500ad3f2aca" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 37199 } } peers { permanent_uuid: "ea739164f1b344099a538c6c91d0ada6" member_type: NON_VOTER last_known_addr { host: "127.29.112.189" port: 45725 } attrs { promote: true } }
I20260811 18:57:59.193553 9753 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea739164f1b344099a538c6c91d0ada6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b994bc87dbdf4d93a9b50500ad3f2aca" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 37199 } } peers { permanent_uuid: "ea739164f1b344099a538c6c91d0ada6" member_type: NON_VOTER last_known_addr { host: "127.29.112.189" port: 45725 } attrs { promote: true } } }
I20260811 18:57:59.193902 9748 sys_catalog.cc:564] T 00000000000000000000000000000000 P ea739164f1b344099a538c6c91d0ada6 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:57:59.193948 9753 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea739164f1b344099a538c6c91d0ada6 [sys.catalog]: This master's current role is: LEARNER
I20260811 18:57:59.204569 9775 catalog_manager.cc:1259] Loaded cluster ID: ddf73710ba7c49fbb2a6a28cae586e9c
I20260811 18:57:59.204689 9775 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ea739164f1b344099a538c6c91d0ada6: loading cluster ID for follower catalog manager: success
I20260811 18:57:59.207360 9775 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ea739164f1b344099a538c6c91d0ada6: acquiring CA information for follower catalog manager: success
I20260811 18:57:59.209218 9775 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ea739164f1b344099a538c6c91d0ada6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 18:57:59.283042 9445 heartbeater.cc:503] Master 127.29.112.190:37199 was elected leader, sending a full tablet report...
W20260811 18:57:59.450073 9619 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:57:59.450446 9619 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:57:59.450659 9619 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:57:59.458453 9619 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:57:59.458627 9619 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:57:59.458750 9619 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:57:59.458856 9619 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:57:59.458933 9619 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:57:59.468811 9619 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:43681
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.29.112.190:37199,127.29.112.189:45725,127.29.112.188:44651
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/krb5kdc/oryx_127.29.112.188.keytab
--principal=oryx/127.29.112.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.188:44651
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.29.112.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:57:59.470633 9619 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:57:59.472034 9619 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:57:59.484313 9784 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:59.484084 9787 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:59.484229 9785 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:57:59.485299 9619 server_base.cc:1034] running on GCE node
I20260811 18:57:59.485999 9619 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:57:59.487344 9619 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:57:59.488565 9619 hybrid_clock.cc:648] HybridClock initialized: now 1786474679488501 us; error 52 us; skew 500 ppm
I20260811 18:57:59.488935 9619 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:57:59 dist-test-slave-h6n0 krb5kdc[8508](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474679, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.112.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:57:59.492323 9619 init.cc:377] Logged in from keytab as oryx/127.29.112.188@KRBTEST.COM (short username oryx)
I20260811 18:57:59.493842 9619 webserver.cc:460] Webserver started at http://127.29.112.188:34115/ using document root <none> and password file <none>
I20260811 18:57:59.494581 9619 fs_manager.cc:359] Metadata directory not provided
I20260811 18:57:59.494711 9619 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:57:59.495019 9619 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:57:59.497900 9619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/master-2/data/instance:
uuid: "4198b17e42c94aac90009c1bc0970564"
format_stamp: "Formatted at 2026-08-11 18:57:59 on dist-test-slave-h6n0"
server_key: "b425f8b0e97817b19e6deafc76cea9c3"
server_key_iv: "1b4d904cd3036be4cc16858ab601a46a"
server_key_version: "encryptionkey@0"
I20260811 18:57:59.498749 9619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/master-2/wal/instance:
uuid: "4198b17e42c94aac90009c1bc0970564"
format_stamp: "Formatted at 2026-08-11 18:57:59 on dist-test-slave-h6n0"
server_key: "b425f8b0e97817b19e6deafc76cea9c3"
server_key_iv: "1b4d904cd3036be4cc16858ab601a46a"
server_key_version: "encryptionkey@0"
I20260811 18:57:59.504827 9619 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 18:57:59.508999 9801 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:59.510859 9619 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 18:57:59.511070 9619 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/master-2/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/master-2/wal
uuid: "4198b17e42c94aac90009c1bc0970564"
format_stamp: "Formatted at 2026-08-11 18:57:59 on dist-test-slave-h6n0"
server_key: "b425f8b0e97817b19e6deafc76cea9c3"
server_key_iv: "1b4d904cd3036be4cc16858ab601a46a"
server_key_version: "encryptionkey@0"
I20260811 18:57:59.511276 9619 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:57:59.526763 9619 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:57:59.531101 9619 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:57:59.531373 9619 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:57:59.552991 9619 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.188:44651
I20260811 18:57:59.553086 9886 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.188:44651 every 8 connection(s)
I20260811 18:57:59.554426 9619 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/master-2/data/info.pb
I20260811 18:57:59.554948 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 9619
I20260811 18:57:59.559176 9887 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:59.566823 9887 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.112.190" port: 37199 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 18:57:59 dist-test-slave-h6n0 krb5kdc[8508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474675, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.29.112.188@KRBTEST.COM
Aug 11 18:57:59 dist-test-slave-h6n0 krb5kdc[8508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474679, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.112.188@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM
I20260811 18:57:59.582638 9891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:59.557411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47572 (local address 127.29.112.188:44651)
0811 18:57:59.558508 (+ 1097us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:59.558520 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:59.558636 (+ 116us) server_negotiation.cc:407] Connection header received
0811 18:57:59.560387 (+ 1751us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:59.560411 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:59.560781 (+ 370us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:59.561170 (+ 389us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:59.563043 (+ 1873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:59.564430 (+ 1387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:59.565783 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:59.566481 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:59.566739 (+ 258us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:59.572587 (+ 5848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:59.572597 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:59.572609 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:59.572655 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:59.576699 (+ 4044us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:59.577616 (+ 917us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:59.577627 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:59.577638 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:59.577893 (+ 255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:59.578371 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:59.578376 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:59.578381 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:59.579399 (+ 1018us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:59.579690 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:59.580478 (+ 788us) server_negotiation.cc:299] Negotiation successful
0811 18:57:59.580910 (+ 432us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":879,"spinlock_wait_cycles":30976,"thread_start_us":635,"threads_started":1}
I20260811 18:57:59.589514 9892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:59.569915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37199 (local address 127.0.0.1:60524)
0811 18:57:59.570783 (+ 868us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:59.570840 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:59.571103 (+ 263us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:59.571962 (+ 859us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:59.571970 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:59.572756 (+ 786us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:57:59.573136 (+ 380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:59.573149 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:59.574438 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:59.574443 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:59.576390 (+ 1947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:59.576422 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:59.577417 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:59.577424 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:59.577694 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:59.578371 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:57:59.578401 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:57:59.581564 (+ 3163us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:57:59.586277 (+ 4713us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:59.586288 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:59.586294 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:59.587016 (+ 722us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:59.587430 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:59.587435 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:59.587439 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:59.587580 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:59.588143 (+ 563us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:57:59.588151 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:57:59.588327 (+ 176us) client_negotiation.cc:769] Sending connection context
0811 18:57:59.588513 (+ 186us) client_negotiation.cc:240] Negotiation successful
0811 18:57:59.588832 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":562,"thread_start_us":420,"threads_started":1}
I20260811 18:57:59.589514 9895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:59.570073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60524 (local address 127.29.112.190:37199)
0811 18:57:59.571379 (+ 1306us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:59.571384 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:59.571399 (+ 15us) server_negotiation.cc:407] Connection header received
0811 18:57:59.571484 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:59.571488 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:59.571571 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:59.571907 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:59.573307 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:59.574283 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:59.576571 (+ 2288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:59.577176 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:59.577354 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:59.581801 (+ 4447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:59.581807 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:59.581812 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:59.581862 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:59.586076 (+ 4214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:59.587168 (+ 1092us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:59.587173 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:59.587177 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:59.587310 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:59.587705 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:59.587710 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:59.587714 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:59.587991 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:59.588172 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:59.588632 (+ 460us) server_negotiation.cc:299] Negotiation successful
0811 18:57:59.588850 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1193,"thread_start_us":1091,"threads_started":1}
I20260811 18:57:59.591859 9887 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.112.189" port: 45725 } has no permanent_uuid. Determining permanent_uuid...
I20260811 18:57:59.598629 9899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:59.579458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.189:45725 (local address 127.0.0.1:56260)
0811 18:57:59.590485 (+ 11027us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:59.590512 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:59.590953 (+ 441us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:59.592253 (+ 1300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:59.592257 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:59.592288 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:57:59.592581 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:59.592598 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:59.594615 (+ 2017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:59.594625 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:59.596625 (+ 2000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:59.596641 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:59.597717 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:59.597724 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:59.597905 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:59.597959 (+ 54us) client_negotiation.cc:769] Sending connection context
0811 18:57:59.598072 (+ 113us) client_negotiation.cc:240] Negotiation successful
0811 18:57:59.598221 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10697,"thread_start_us":11623,"threads_started":1}
I20260811 18:57:59.598685 9900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:59.579656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56260 (local address 127.29.112.189:45725)
0811 18:57:59.591741 (+ 12085us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:59.591747 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:59.591765 (+ 18us) server_negotiation.cc:407] Connection header received
0811 18:57:59.591841 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:59.591846 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:59.591941 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:59.592211 (+ 270us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:57:59.592732 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:59.594305 (+ 1573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:59.596788 (+ 2483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:59.597508 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:59.597724 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:59.597915 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:59.598106 (+ 191us) server_negotiation.cc:299] Negotiation successful
0811 18:57:59.598239 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11947,"thread_start_us":11810,"threads_started":1}
I20260811 18:57:59.599762 9705 raft_consensus.cc:1203] T 00000000000000000000000000000000 P ea739164f1b344099a538c6c91d0ada6 [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
I20260811 18:57:59.600390 9753 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea739164f1b344099a538c6c91d0ada6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b994bc87dbdf4d93a9b50500ad3f2aca. Latest consensus state: current_term: 1 leader_uuid: "b994bc87dbdf4d93a9b50500ad3f2aca" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b994bc87dbdf4d93a9b50500ad3f2aca" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 37199 } } peers { permanent_uuid: "ea739164f1b344099a538c6c91d0ada6" member_type: NON_VOTER last_known_addr { host: "127.29.112.189" port: 45725 } attrs { promote: true } } }
I20260811 18:57:59.600675 9753 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea739164f1b344099a538c6c91d0ada6 [sys.catalog]: This master's current role is: LEARNER
Aug 11 18:57:59 dist-test-slave-h6n0 krb5kdc[8508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474679, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.112.188@KRBTEST.COM for oryx/127.29.112.189@KRBTEST.COM
I20260811 18:57:59.607213 9902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:59.592733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56266 (local address 127.29.112.189:45725)
0811 18:57:59.593472 (+ 739us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:59.593478 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:59.593491 (+ 13us) server_negotiation.cc:407] Connection header received
0811 18:57:59.593525 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:59.593528 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:59.593576 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:59.593916 (+ 340us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:59.595158 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:59.595909 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:59.597434 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:59.597899 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:59.598030 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:59.601862 (+ 3832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:59.601867 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:59.601871 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:59.601899 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:59.604349 (+ 2450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:59.605009 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:59.605016 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:59.605020 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:59.605197 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:59.605580 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:59.605586 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:59.605589 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:59.605860 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:59.606009 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:59.606760 (+ 751us) server_negotiation.cc:299] Negotiation successful
0811 18:57:59.606944 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":595,"thread_start_us":455,"threads_started":1}
I20260811 18:57:59.607272 9892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:59.592756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.189:45725 (local address 127.0.0.1:56266)
0811 18:57:59.592939 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:59.592957 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:59.593075 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:59.593937 (+ 862us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:59.593941 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:59.594671 (+ 730us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:57:59.594946 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:59.594964 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:59.596070 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:59.596077 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:59.597275 (+ 1198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:59.597289 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:59.598060 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:59.598067 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:59.598274 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:59.599106 (+ 832us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:57:59.599123 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:57:59.601644 (+ 2521us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:57:59.604475 (+ 2831us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:59.604482 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:59.604486 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:59.604893 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:59.605306 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:59.605311 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:59.605314 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:59.605467 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:59.605977 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:57:59.605982 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:57:59.606282 (+ 300us) client_negotiation.cc:769] Sending connection context
0811 18:57:59.606612 (+ 330us) client_negotiation.cc:240] Negotiation successful
0811 18:57:59.606965 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260811 18:57:59.608960 9887 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.112.188" port: 44651 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 18:57:59 dist-test-slave-h6n0 krb5kdc[8508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474679, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.112.188@KRBTEST.COM for oryx/127.29.112.188@KRBTEST.COM
I20260811 18:57:59.622520 9892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:59.610326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.188:44651 (local address 127.0.0.1:47582)
0811 18:57:59.610569 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:59.610595 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:59.610732 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:59.611031 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:59.611035 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:59.611440 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:57:59.611582 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:59.611590 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:59.612573 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:59.612581 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:59.613401 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:59.613412 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:59.614090 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:59.614097 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:59.614293 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:59.614912 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:57:59.614930 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:57:59.617485 (+ 2555us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:57:59.620428 (+ 2943us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:59.620432 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:59.620435 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:59.620777 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:59.621107 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:59.621111 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:59.621113 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:59.621262 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:59.621719 (+ 457us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:57:59.621723 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:57:59.621861 (+ 138us) client_negotiation.cc:769] Sending connection context
0811 18:57:59.621984 (+ 123us) client_negotiation.cc:240] Negotiation successful
0811 18:57:59.622157 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83}
I20260811 18:57:59.622818 9891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:59.610126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47582 (local address 127.29.112.188:44651)
0811 18:57:59.610476 (+ 350us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:59.610482 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:59.610666 (+ 184us) server_negotiation.cc:407] Connection header received
0811 18:57:59.610849 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:59.610853 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:59.610901 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:59.611005 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:59.611714 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:59.612450 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:59.613543 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:59.613928 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:59.614060 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:59.617701 (+ 3641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:59.617706 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:59.617709 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:59.617737 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:59.620297 (+ 2560us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:59.620875 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:59.620879 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:59.620881 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:59.621011 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:59.621366 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:59.621369 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:59.621371 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:59.621617 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:59.621719 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:59.622109 (+ 390us) server_negotiation.cc:299] Negotiation successful
0811 18:57:59.622279 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184}
I20260811 18:57:59.627517 9887 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4198b17e42c94aac90009c1bc0970564: Bootstrap starting.
I20260811 18:57:59.630523 9887 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4198b17e42c94aac90009c1bc0970564: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:59.631580 9887 log.cc:824] T 00000000000000000000000000000000 P 4198b17e42c94aac90009c1bc0970564: Log is configured to *not* fsync() on all Append() calls
I20260811 18:57:59.634021 9887 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4198b17e42c94aac90009c1bc0970564: No bootstrap required, opened a new log
I20260811 18:57:59.642644 9887 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4198b17e42c94aac90009c1bc0970564 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b994bc87dbdf4d93a9b50500ad3f2aca" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 37199 } } peers { permanent_uuid: "ea739164f1b344099a538c6c91d0ada6" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 45725 } } peers { permanent_uuid: "4198b17e42c94aac90009c1bc0970564" member_type: VOTER last_known_addr { host: "127.29.112.188" port: 44651 } }
I20260811 18:57:59.643007 9887 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4198b17e42c94aac90009c1bc0970564 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:59.643154 9887 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4198b17e42c94aac90009c1bc0970564 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4198b17e42c94aac90009c1bc0970564, State: Initialized, Role: FOLLOWER
I20260811 18:57:59.643759 9887 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4198b17e42c94aac90009c1bc0970564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b994bc87dbdf4d93a9b50500ad3f2aca" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 37199 } } peers { permanent_uuid: "ea739164f1b344099a538c6c91d0ada6" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 45725 } } peers { permanent_uuid: "4198b17e42c94aac90009c1bc0970564" member_type: VOTER last_known_addr { host: "127.29.112.188" port: 44651 } }
I20260811 18:57:59.644694 9905 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4198b17e42c94aac90009c1bc0970564 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b994bc87dbdf4d93a9b50500ad3f2aca" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 37199 } } peers { permanent_uuid: "ea739164f1b344099a538c6c91d0ada6" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 45725 } } peers { permanent_uuid: "4198b17e42c94aac90009c1bc0970564" member_type: VOTER last_known_addr { host: "127.29.112.188" port: 44651 } } }
I20260811 18:57:59.645008 9905 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4198b17e42c94aac90009c1bc0970564 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:57:59.645773 9887 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4198b17e42c94aac90009c1bc0970564 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 18:57:59.655524 9935 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4198b17e42c94aac90009c1bc0970564: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:57:59.655658 9935 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:57:59.660581 9619 master_runner.cc:428] Detected that this master 4198b17e42c94aac90009c1bc0970564 is joining an existing cluster
I20260811 18:57:59.660744 9619 master_runner.cc:432] Initiating AddMaster RPC to add 127.29.112.188:44651
I20260811 18:57:59.685181 9902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:59.666107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56274 (local address 127.29.112.189:45725)
0811 18:57:59.666404 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:59.666413 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:59.667496 (+ 1083us) server_negotiation.cc:407] Connection header received
0811 18:57:59.668021 (+ 525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:59.668028 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:59.668136 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:59.668220 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:59.670028 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:59.671330 (+ 1302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:59.673586 (+ 2256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:59.674250 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:59.674377 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:59.677243 (+ 2866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:59.677251 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:59.677256 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:59.677300 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:59.681375 (+ 4075us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:59.682523 (+ 1148us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:59.682559 (+ 36us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:59.682565 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:59.682732 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:59.683275 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:59.683281 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:59.683285 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:59.683648 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:59.683782 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:59.684564 (+ 782us) server_negotiation.cc:299] Negotiation successful
0811 18:57:59.684810 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260811 18:57:59.686709 9942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:59.665780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.189:45725 (local address 127.0.0.1:56274)
0811 18:57:59.667382 (+ 1602us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:59.667403 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:59.667801 (+ 398us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:59.668473 (+ 672us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:59.668479 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:59.669297 (+ 818us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:57:59.669752 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:59.669827 (+ 75us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:59.671622 (+ 1795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:59.671628 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:59.673391 (+ 1763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:59.673409 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:59.674734 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:59.674741 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:59.674922 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:59.675753 (+ 831us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:57:59.675781 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:57:59.677028 (+ 1247us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:57:59.681578 (+ 4550us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:59.681589 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:59.681596 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:59.682358 (+ 762us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:59.682885 (+ 527us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:59.682890 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:59.682897 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:59.683120 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:59.683887 (+ 767us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:57:59.683892 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:57:59.684174 (+ 282us) client_negotiation.cc:769] Sending connection context
0811 18:57:59.685769 (+ 1595us) client_negotiation.cc:240] Negotiation successful
0811 18:57:59.686120 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1417,"thread_start_us":524,"threads_started":1}
I20260811 18:57:59.690124 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 8697
I20260811 18:57:59.704299 9891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:59.667367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47596 (local address 127.29.112.188:44651)
0811 18:57:59.667572 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:59.667577 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:59.687384 (+ 19807us) server_negotiation.cc:407] Connection header received
0811 18:57:59.687800 (+ 416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:59.687808 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:59.687938 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:59.688103 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:59.690304 (+ 2201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:59.691426 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:59.693283 (+ 1857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:59.694146 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:59.694297 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:59.696905 (+ 2608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:59.696911 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:59.696916 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:59.696949 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:59.700632 (+ 3683us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:59.701577 (+ 945us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:59.701582 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:59.701586 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:59.701713 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:59.702144 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:59.702154 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:59.702158 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:59.702631 (+ 473us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:59.702792 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:59.703661 (+ 869us) server_negotiation.cc:299] Negotiation successful
0811 18:57:59.703885 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 18:57:59.705312 9895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:59.665750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60538 (local address 127.29.112.190:37199)
0811 18:57:59.666018 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:59.666027 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:59.681444 (+ 15417us) server_negotiation.cc:407] Connection header received
0811 18:57:59.682073 (+ 629us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:59.682083 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:59.682204 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:59.682295 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:59.684289 (+ 1994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:59.685818 (+ 1529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:59.690936 (+ 5118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:59.691570 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:59.691670 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:59.695826 (+ 4156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:59.695835 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:59.695843 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:59.695912 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:59.700469 (+ 4557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:59.701731 (+ 1262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:59.701739 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:59.701746 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:59.701952 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:59.702573 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:59.702581 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:59.702587 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:59.703073 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:59.703245 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:59.704402 (+ 1157us) server_negotiation.cc:299] Negotiation successful
0811 18:57:59.704992 (+ 590us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260811 18:57:59.705359 9942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:59.668012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.188:44651 (local address 127.0.0.1:47596)
0811 18:57:59.687291 (+ 19279us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:59.687311 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:59.687466 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:59.688419 (+ 953us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:59.688424 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:59.689867 (+ 1443us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:57:59.690066 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:59.690080 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:59.691685 (+ 1605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:59.691690 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:59.693126 (+ 1436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:59.693141 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:59.694313 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:59.694317 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:59.694850 (+ 533us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:59.695574 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:57:59.695606 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:57:59.696721 (+ 1115us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:57:59.700787 (+ 4066us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:59.700792 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:59.700795 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:59.701449 (+ 654us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:59.701840 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:59.701845 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:59.701848 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:59.702031 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:59.702841 (+ 810us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:57:59.702846 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:57:59.703314 (+ 468us) client_negotiation.cc:769] Sending connection context
0811 18:57:59.704577 (+ 1263us) client_negotiation.cc:240] Negotiation successful
0811 18:57:59.704849 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19169,"mutex_wait_us":9,"thread_start_us":13463,"threads_started":1}
I20260811 18:57:59.706765 9945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:59.667383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37199 (local address 127.0.0.1:60538)
0811 18:57:59.681286 (+ 13903us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:59.681313 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:59.681874 (+ 561us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:59.682593 (+ 719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:59.682601 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:59.683559 (+ 958us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:57:59.684050 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:59.684069 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:59.686076 (+ 2007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:59.686085 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:59.690742 (+ 4657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:59.690764 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:59.692024 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:59.692033 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:59.692299 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:59.693208 (+ 909us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:57:59.693241 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:57:59.695554 (+ 2313us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:57:59.700692 (+ 5138us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:59.700700 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:59.700706 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:59.701561 (+ 855us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:59.702128 (+ 567us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:59.702136 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:59.702142 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:59.702394 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:59.703352 (+ 958us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:57:59.703356 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:57:59.703639 (+ 283us) client_negotiation.cc:769] Sending connection context
0811 18:57:59.705787 (+ 2148us) client_negotiation.cc:240] Negotiation successful
0811 18:57:59.706203 (+ 416us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13689,"mutex_wait_us":1,"thread_start_us":21987,"threads_started":1}
Aug 11 18:57:59 dist-test-slave-h6n0 krb5kdc[8508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474675, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.29.112.129@KRBTEST.COM
Aug 11 18:57:59 dist-test-slave-h6n0 krb5kdc[8508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474676, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.112.190@KRBTEST.COM for oryx/127.29.112.188@KRBTEST.COM
I20260811 18:57:59.727808 9946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:59.691945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35794 (local address 127.29.112.129:35525)
0811 18:57:59.704657 (+ 12712us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:59.704671 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:59.704712 (+ 41us) server_negotiation.cc:407] Connection header received
0811 18:57:59.704944 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:59.704957 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:59.705278 (+ 321us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:59.707580 (+ 2302us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:59.707793 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:59.709449 (+ 1656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:59.710927 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:59.711652 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:59.711778 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:59.718092 (+ 6314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:59.718110 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:59.718130 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:59.718180 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:59.723576 (+ 5396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:59.724583 (+ 1007us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:59.724593 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:59.724596 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:59.724783 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:59.725323 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:59.725331 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:59.725336 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:59.725826 (+ 490us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:59.726857 (+ 1031us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:59.727072 (+ 215us) server_negotiation.cc:299] Negotiation successful
0811 18:57:59.727370 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12445,"thread_start_us":13482,"threads_started":1}
I20260811 18:57:59.729559 8927 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:57:59.730476 9891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:59.712735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47612 (local address 127.29.112.188:44651)
0811 18:57:59.712890 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:59.712898 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:59.712921 (+ 23us) server_negotiation.cc:407] Connection header received
0811 18:57:59.713009 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:59.713014 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:59.713097 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:59.713201 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:59.714221 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:59.715372 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:59.717197 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:59.717956 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:59.718098 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:59.723471 (+ 5373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:59.723479 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:59.723485 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:59.723524 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:59.727333 (+ 3809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:59.728099 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:59.728104 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:59.728107 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:59.728264 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:59.728661 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:59.728666 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:59.728670 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:59.728944 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:59.729106 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:59.729866 (+ 760us) server_negotiation.cc:299] Negotiation successful
0811 18:57:59.730110 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 18:57:59.730476 9899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:59.712385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.188:44651 (local address 127.0.0.1:47612)
0811 18:57:59.712605 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:59.712627 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:59.712745 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:59.713261 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:59.713267 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:59.713799 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:57:59.714015 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:59.714030 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:59.715574 (+ 1544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:59.715581 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:59.717025 (+ 1444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:59.717043 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:59.718495 (+ 1452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:59.718501 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:59.718712 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:59.719392 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:57:59.719425 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:57:59.723186 (+ 3761us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:57:59.727488 (+ 4302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:59.727495 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:59.727499 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:59.727922 (+ 423us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:59.728391 (+ 469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:59.728397 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:59.728400 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:59.728546 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:59.729076 (+ 530us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:57:59.729081 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:57:59.729275 (+ 194us) client_negotiation.cc:769] Sending connection context
0811 18:57:59.729431 (+ 156us) client_negotiation.cc:240] Negotiation successful
0811 18:57:59.729755 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260811 18:57:59.732369 8599 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.29.112.188:44651: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "4198b17e42c94aac90009c1bc0970564"
member_type: NON_VOTER
last_known_addr {
host: "127.29.112.188"
port: 44651
}
attrs {
promote: true
}
}
dest_uuid: "b994bc87dbdf4d93a9b50500ad3f2aca"
cas_config_opid_index: 5
I20260811 18:57:59.733429 8599 consensus_queue.cc:237] T 00000000000000000000000000000000 P b994bc87dbdf4d93a9b50500ad3f2aca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b994bc87dbdf4d93a9b50500ad3f2aca" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 37199 } } peers { permanent_uuid: "ea739164f1b344099a538c6c91d0ada6" member_type: NON_VOTER last_known_addr { host: "127.29.112.189" port: 45725 } attrs { promote: true } } peers { permanent_uuid: "4198b17e42c94aac90009c1bc0970564" member_type: NON_VOTER last_known_addr { host: "127.29.112.188" port: 44651 } attrs { promote: true } }
I20260811 18:57:59.736754 9705 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ea739164f1b344099a538c6c91d0ada6 [term 1 LEARNER]: Refusing update from remote peer b994bc87dbdf4d93a9b50500ad3f2aca: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260811 18:57:59.738971 9843 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4198b17e42c94aac90009c1bc0970564 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:59.744112 9896 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b994bc87dbdf4d93a9b50500ad3f2aca [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 4198b17e42c94aac90009c1bc0970564 (127.29.112.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b994bc87dbdf4d93a9b50500ad3f2aca" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 37199 } } peers { permanent_uuid: "ea739164f1b344099a538c6c91d0ada6" member_type: NON_VOTER last_known_addr { host: "127.29.112.189" port: 45725 } attrs { promote: true } } peers { permanent_uuid: "4198b17e42c94aac90009c1bc0970564" member_type: NON_VOTER last_known_addr { host: "127.29.112.188" port: 44651 } attrs { promote: true } } }
I20260811 18:57:59.747494 9843 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4198b17e42c94aac90009c1bc0970564 [term 1 FOLLOWER]: Refusing update from remote peer b994bc87dbdf4d93a9b50500ad3f2aca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260811 18:57:59.752880 9954 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b994bc87dbdf4d93a9b50500ad3f2aca [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea739164f1b344099a538c6c91d0ada6" member_type: NON_VOTER last_known_addr { host: "127.29.112.189" port: 45725 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260811 18:57:59.758455 9955 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b994bc87dbdf4d93a9b50500ad3f2aca [LEADER]: Connected to new peer: Peer: permanent_uuid: "4198b17e42c94aac90009c1bc0970564" member_type: NON_VOTER last_known_addr { host: "127.29.112.188" port: 44651 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:59.759529 9705 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ea739164f1b344099a538c6c91d0ada6 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 4198b17e42c94aac90009c1bc0970564 (127.29.112.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b994bc87dbdf4d93a9b50500ad3f2aca" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 37199 } } peers { permanent_uuid: "ea739164f1b344099a538c6c91d0ada6" member_type: NON_VOTER last_known_addr { host: "127.29.112.189" port: 45725 } attrs { promote: true } } peers { permanent_uuid: "4198b17e42c94aac90009c1bc0970564" member_type: NON_VOTER last_known_addr { host: "127.29.112.188" port: 44651 } attrs { promote: true } } }
I20260811 18:57:59.764902 9753 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea739164f1b344099a538c6c91d0ada6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b994bc87dbdf4d93a9b50500ad3f2aca" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b994bc87dbdf4d93a9b50500ad3f2aca" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 37199 } } peers { permanent_uuid: "ea739164f1b344099a538c6c91d0ada6" member_type: NON_VOTER last_known_addr { host: "127.29.112.189" port: 45725 } attrs { promote: true } } peers { permanent_uuid: "4198b17e42c94aac90009c1bc0970564" member_type: NON_VOTER last_known_addr { host: "127.29.112.188" port: 44651 } attrs { promote: true } } }
I20260811 18:57:59.765249 9753 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea739164f1b344099a538c6c91d0ada6 [sys.catalog]: This master's current role is: LEARNER
I20260811 18:57:59.765612 9896 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.29.112.188:44651
I20260811 18:57:59.766346 9619 master.cc:537] Master@127.29.112.188:44651 shutting down...
I20260811 18:57:59.767549 9954 sys_catalog.cc:455] T 00000000000000000000000000000000 P b994bc87dbdf4d93a9b50500ad3f2aca [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "b994bc87dbdf4d93a9b50500ad3f2aca" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b994bc87dbdf4d93a9b50500ad3f2aca" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 37199 } } peers { permanent_uuid: "ea739164f1b344099a538c6c91d0ada6" member_type: NON_VOTER last_known_addr { host: "127.29.112.189" port: 45725 } attrs { promote: true } } peers { permanent_uuid: "4198b17e42c94aac90009c1bc0970564" member_type: NON_VOTER last_known_addr { host: "127.29.112.188" port: 44651 } attrs { promote: true } } }
I20260811 18:57:59.767968 9954 sys_catalog.cc:458] T 00000000000000000000000000000000 P b994bc87dbdf4d93a9b50500ad3f2aca [sys.catalog]: This master's current role is: LEADER
I20260811 18:57:59.769840 9955 raft_consensus.cc:1050] T 00000000000000000000000000000000 P b994bc87dbdf4d93a9b50500ad3f2aca: attempting to promote NON_VOTER ea739164f1b344099a538c6c91d0ada6 to VOTER
I20260811 18:57:59.770828 9955 consensus_queue.cc:237] T 00000000000000000000000000000000 P b994bc87dbdf4d93a9b50500ad3f2aca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b994bc87dbdf4d93a9b50500ad3f2aca" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 37199 } } peers { permanent_uuid: "ea739164f1b344099a538c6c91d0ada6" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 45725 } attrs { promote: false } } peers { permanent_uuid: "4198b17e42c94aac90009c1bc0970564" member_type: NON_VOTER last_known_addr { host: "127.29.112.188" port: 44651 } attrs { promote: true } }
I20260811 18:57:59.780378 9705 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ea739164f1b344099a538c6c91d0ada6 [term 1 LEARNER]: Refusing update from remote peer b994bc87dbdf4d93a9b50500ad3f2aca: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260811 18:57:59.781030 9961 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b994bc87dbdf4d93a9b50500ad3f2aca [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea739164f1b344099a538c6c91d0ada6" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 45725 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260811 18:57:59.783577 9808 tablet.cc:2369] T 00000000000000000000000000000000 P 4198b17e42c94aac90009c1bc0970564: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:57:59.784292 9961 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b994bc87dbdf4d93a9b50500ad3f2aca [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, ea739164f1b344099a538c6c91d0ada6 (127.29.112.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b994bc87dbdf4d93a9b50500ad3f2aca" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 37199 } } peers { permanent_uuid: "ea739164f1b344099a538c6c91d0ada6" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 45725 } attrs { promote: false } } peers { permanent_uuid: "4198b17e42c94aac90009c1bc0970564" member_type: NON_VOTER last_known_addr { host: "127.29.112.188" port: 44651 } attrs { promote: true } } }
I20260811 18:57:59.787120 9704 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ea739164f1b344099a538c6c91d0ada6 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, ea739164f1b344099a538c6c91d0ada6 (127.29.112.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b994bc87dbdf4d93a9b50500ad3f2aca" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 37199 } } peers { permanent_uuid: "ea739164f1b344099a538c6c91d0ada6" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 45725 } attrs { promote: false } } peers { permanent_uuid: "4198b17e42c94aac90009c1bc0970564" member_type: NON_VOTER last_known_addr { host: "127.29.112.188" port: 44651 } attrs { promote: true } } }
I20260811 18:57:59.793359 9753 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea739164f1b344099a538c6c91d0ada6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b994bc87dbdf4d93a9b50500ad3f2aca" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b994bc87dbdf4d93a9b50500ad3f2aca" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 37199 } } peers { permanent_uuid: "ea739164f1b344099a538c6c91d0ada6" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 45725 } attrs { promote: false } } peers { permanent_uuid: "4198b17e42c94aac90009c1bc0970564" member_type: NON_VOTER last_known_addr { host: "127.29.112.188" port: 44651 } attrs { promote: true } } }
I20260811 18:57:59.793622 9753 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea739164f1b344099a538c6c91d0ada6 [sys.catalog]: This master's current role is: FOLLOWER
W20260811 18:57:59.794068 8568 consensus_peers.cc:487] T 00000000000000000000000000000000 P b994bc87dbdf4d93a9b50500ad3f2aca -> Peer 4198b17e42c94aac90009c1bc0970564 (127.29.112.188:44651): Couldn't send request to peer 4198b17e42c94aac90009c1bc0970564. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260811 18:57:59.794832 9896 sys_catalog.cc:455] T 00000000000000000000000000000000 P b994bc87dbdf4d93a9b50500ad3f2aca [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "b994bc87dbdf4d93a9b50500ad3f2aca" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b994bc87dbdf4d93a9b50500ad3f2aca" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 37199 } } peers { permanent_uuid: "ea739164f1b344099a538c6c91d0ada6" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 45725 } attrs { promote: false } } peers { permanent_uuid: "4198b17e42c94aac90009c1bc0970564" member_type: NON_VOTER last_known_addr { host: "127.29.112.188" port: 44651 } attrs { promote: true } } }
I20260811 18:57:59.795084 9896 sys_catalog.cc:458] T 00000000000000000000000000000000 P b994bc87dbdf4d93a9b50500ad3f2aca [sys.catalog]: This master's current role is: LEADER
I20260811 18:57:59.795447 9961 sys_catalog.cc:455] T 00000000000000000000000000000000 P b994bc87dbdf4d93a9b50500ad3f2aca [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b994bc87dbdf4d93a9b50500ad3f2aca" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b994bc87dbdf4d93a9b50500ad3f2aca" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 37199 } } peers { permanent_uuid: "ea739164f1b344099a538c6c91d0ada6" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 45725 } attrs { promote: false } } peers { permanent_uuid: "4198b17e42c94aac90009c1bc0970564" member_type: NON_VOTER last_known_addr { host: "127.29.112.188" port: 44651 } attrs { promote: true } } }
I20260811 18:57:59.795737 9961 sys_catalog.cc:458] T 00000000000000000000000000000000 P b994bc87dbdf4d93a9b50500ad3f2aca [sys.catalog]: This master's current role is: LEADER
I20260811 18:57:59.796825 9843 raft_consensus.cc:858] T 00000000000000000000000000000000 P 4198b17e42c94aac90009c1bc0970564 [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b994bc87dbdf4d93a9b50500ad3f2aca" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 37199 } } peers { permanent_uuid: "ea739164f1b344099a538c6c91d0ada6" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 45725 } } peers { permanent_uuid: "4198b17e42c94aac90009c1bc0970564" member_type: VOTER last_known_addr { host: "127.29.112.188" port: 44651 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b994bc87dbdf4d93a9b50500ad3f2aca" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 37199 } } peers { permanent_uuid: "ea739164f1b344099a538c6c91d0ada6" member_type: NON_VOTER last_known_addr { host: "127.29.112.189" port: 45725 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b994bc87dbdf4d93a9b50500ad3f2aca" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 37199 } } peers { permanent_uuid: "ea739164f1b344099a538c6c91d0ada6" member_type: NON_VOTER last_known_addr { host: "127.29.112.189" port: 45725 } attrs { promote: true } } peers { permanent_uuid: "4198b17e42c94aac90009c1bc0970564" member_type: NON_VOTER last_known_addr { host: "127.29.112.188" port: 44651 } attrs { promote: true } }
W20260811 18:57:59.797116 9843 raft_consensus.cc:1565] T 00000000000000000000000000000000 P 4198b17e42c94aac90009c1bc0970564 [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 6' and following 0 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b994bc87dbdf4d93a9b50500ad3f2aca" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 37199 } } peers { permanent_uuid: "ea739164f1b344099a538c6c91d0ada6" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 45725 } } peers { permanent_uuid: "4198b17e42c94aac90009c1bc0970564" member_type: VOTER last_known_addr { host: "127.29.112.188" port: 44651 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b994bc87dbdf4d93a9b50500ad3f2aca" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 37199 } } peers { permanent_uuid: "ea739164f1b344099a538c6c91d0ada6" member_type: NON_VOTER last_known_addr { host: "127.29.112.189" port: 45725 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b994bc87dbdf4d93a9b50500ad3f2aca" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 37199 } } peers { permanent_uuid: "ea739164f1b344099a538c6c91d0ada6" member_type: NON_VOTER last_known_addr { host: "127.29.112.189" port: 45725 } attrs { promote: true } } peers { permanent_uuid: "4198b17e42c94aac90009c1bc0970564" member_type: NON_VOTER last_known_addr { host: "127.29.112.188" port: 44651 } attrs { promote: true } }
I20260811 18:57:59.803112 9843 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 4198b17e42c94aac90009c1bc0970564 [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER ea739164f1b344099a538c6c91d0ada6 (127.29.112.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b994bc87dbdf4d93a9b50500ad3f2aca" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 37199 } } peers { permanent_uuid: "ea739164f1b344099a538c6c91d0ada6" member_type: NON_VOTER last_known_addr { host: "127.29.112.189" port: 45725 } attrs { promote: true } } }
I20260811 18:57:59.840107 9969 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4198b17e42c94aac90009c1bc0970564 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b994bc87dbdf4d93a9b50500ad3f2aca" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b994bc87dbdf4d93a9b50500ad3f2aca" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 37199 } } peers { permanent_uuid: "ea739164f1b344099a538c6c91d0ada6" member_type: NON_VOTER last_known_addr { host: "127.29.112.189" port: 45725 } attrs { promote: true } } }
I20260811 18:57:59.840459 9969 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4198b17e42c94aac90009c1bc0970564 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260811 18:57:59.840801 9905 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4198b17e42c94aac90009c1bc0970564 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b994bc87dbdf4d93a9b50500ad3f2aca. Latest consensus state: current_term: 1 leader_uuid: "b994bc87dbdf4d93a9b50500ad3f2aca" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b994bc87dbdf4d93a9b50500ad3f2aca" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 37199 } } peers { permanent_uuid: "ea739164f1b344099a538c6c91d0ada6" member_type: NON_VOTER last_known_addr { host: "127.29.112.189" port: 45725 } attrs { promote: true } } }
I20260811 18:57:59.840999 9905 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4198b17e42c94aac90009c1bc0970564 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260811 18:57:59.841447 9975 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4198b17e42c94aac90009c1bc0970564 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b994bc87dbdf4d93a9b50500ad3f2aca" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b994bc87dbdf4d93a9b50500ad3f2aca" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 37199 } } peers { permanent_uuid: "ea739164f1b344099a538c6c91d0ada6" member_type: NON_VOTER last_known_addr { host: "127.29.112.189" port: 45725 } attrs { promote: true } } }
I20260811 18:57:59.841730 9975 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4198b17e42c94aac90009c1bc0970564 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260811 18:57:59.842293 9619 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 4198b17e42c94aac90009c1bc0970564 [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
I20260811 18:57:59.842803 9619 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 4198b17e42c94aac90009c1bc0970564 [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20260811 18:57:59.842979 9619 tablet_replica.cc:330] T 00000000000000000000000000000000 P 4198b17e42c94aac90009c1bc0970564: stopping tablet replica
I20260811 18:57:59.861778 9619 master.cc:552] Master@127.29.112.188:44651 shutdown complete.
I20260811 18:57:59.861920 9619 master_runner.cc:305] Clearing existing system tablet
I20260811 18:57:59.862305 9619 fs_manager.cc:359] Metadata directory not provided
I20260811 18:57:59.862437 9619 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:57:59.867254 9619 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.004s
I20260811 18:57:59.869019 9981 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:59.869509 9619 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:57:59.869788 9619 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260811 18:57:59.869890 9619 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/master-2/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/master-2/wal
uuid: "4198b17e42c94aac90009c1bc0970564"
format_stamp: "Formatted at 2026-08-11 18:57:59 on dist-test-slave-h6n0"
server_key: "b425f8b0e97817b19e6deafc76cea9c3"
server_key_iv: "1b4d904cd3036be4cc16858ab601a46a"
server_key_version: "encryptionkey@0"
I20260811 18:57:59.875046 9619 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 4198b17e42c94aac90009c1bc0970564: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:59.882125 9619 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 4198b17e42c94aac90009c1bc0970564: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260811 18:57:59.882298 9619 log.cc:1196] T 00000000000000000000000000000000 P 4198b17e42c94aac90009c1bc0970564: Deleting WAL directory at /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260811 18:57:59.882895 9619 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 4198b17e42c94aac90009c1bc0970564: Deleting consensus metadata
I20260811 18:57:59.883204 9619 master_runner.cc:315] Copying system tablet from 127.29.112.190:37199
I20260811 18:57:59.886328 9619 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 4198b17e42c94aac90009c1bc0970564: tablet copy: Beginning tablet copy session from remote peer at address 127.29.112.190:37199
I20260811 18:57:59.898190 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 8697
I20260811 18:57:59.905689 9895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:59.887387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60540 (local address 127.29.112.190:37199)
0811 18:57:59.887633 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:59.887640 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:59.888336 (+ 696us) server_negotiation.cc:407] Connection header received
0811 18:57:59.888547 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:59.888554 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:59.888781 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:59.888923 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:59.891168 (+ 2245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:59.892348 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:59.894854 (+ 2506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:59.895546 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:59.895679 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:59.898195 (+ 2516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:59.898202 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:59.898207 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:59.898245 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:59.902166 (+ 3921us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:59.903174 (+ 1008us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:59.903180 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:59.903184 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:59.903338 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:59.903856 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:59.903861 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:59.903864 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:59.904229 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:59.904389 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:59.904981 (+ 592us) server_negotiation.cc:299] Negotiation successful
0811 18:57:59.905232 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121}
I20260811 18:57:59.905869 9992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:59.887346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37199 (local address 127.0.0.1:60540)
0811 18:57:59.888155 (+ 809us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:59.888176 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:59.888365 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:59.889723 (+ 1358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:59.889731 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:59.890441 (+ 710us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:57:59.890953 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:59.890971 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:59.892540 (+ 1569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:59.892549 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:59.894678 (+ 2129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:59.894699 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:59.895747 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:59.895756 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:59.896016 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:59.896890 (+ 874us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:57:59.896922 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:57:59.898005 (+ 1083us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:57:59.902308 (+ 4303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:59.902317 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:59.902324 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:59.903040 (+ 716us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:59.903472 (+ 432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:59.903480 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:59.903486 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:59.903709 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:59.904374 (+ 665us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:57:59.904383 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:57:59.904623 (+ 240us) client_negotiation.cc:769] Sending connection context
0811 18:57:59.904780 (+ 157us) client_negotiation.cc:240] Negotiation successful
0811 18:57:59.905185 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":632,"thread_start_us":443,"threads_started":1}
I20260811 18:57:59.906839 8624 tablet_copy_service.cc:140] P b994bc87dbdf4d93a9b50500ad3f2aca: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 4198b17e42c94aac90009c1bc0970564 ({username='oryx', principal='oryx/127.29.112.188@KRBTEST.COM'} at 127.0.0.1:60540)
I20260811 18:57:59.907024 8624 tablet_copy_service.cc:161] P b994bc87dbdf4d93a9b50500ad3f2aca: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 4198b17e42c94aac90009c1bc0970564 at {username='oryx', principal='oryx/127.29.112.188@KRBTEST.COM'} at 127.0.0.1:60540: session id = 4198b17e42c94aac90009c1bc0970564-00000000000000000000000000000000
I20260811 18:57:59.911458 8624 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P b994bc87dbdf4d93a9b50500ad3f2aca: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 18:57:59.913564 9619 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260811 18:57:59.921676 9619 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 4198b17e42c94aac90009c1bc0970564: tablet copy: Starting download of 0 data blocks...
I20260811 18:57:59.922051 9619 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 4198b17e42c94aac90009c1bc0970564: tablet copy: Starting download of 1 WAL segments...
I20260811 18:57:59.925074 9619 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 4198b17e42c94aac90009c1bc0970564: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 18:57:59.931056 8624 tablet_copy_service.cc:342] P b994bc87dbdf4d93a9b50500ad3f2aca: Request end of tablet copy session 4198b17e42c94aac90009c1bc0970564-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.29.112.188@KRBTEST.COM'} at 127.0.0.1:60540
I20260811 18:57:59.931195 8624 tablet_copy_service.cc:433] P b994bc87dbdf4d93a9b50500ad3f2aca: ending tablet copy session 4198b17e42c94aac90009c1bc0970564-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 4198b17e42c94aac90009c1bc0970564
I20260811 18:57:59.931595 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.129:35525
--local_ip_for_outbound_sockets=127.29.112.129
--tserver_master_addrs=127.29.112.190:37199
--webserver_port=37475
--webserver_interface=127.29.112.129
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/krb5kdc/oryx_127.29.112.129.keytab
--principal=oryx/127.29.112.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:43681
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.29.112.190:37199,127.29.112.189:45725
--tserver_master_addrs=127.29.112.190:37199,127.29.112.189:45725,127.29.112.188:44651 with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 18:57:59.945143 9619 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:57:59.948487 9995 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:59.948690 10000 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:57:59.953207 9619 server_base.cc:1034] running on GCE node
W20260811 18:57:59.955567 10005 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:57:59.956054 9619 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:57:59.956825 9619 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:57:59.958067 9619 hybrid_clock.cc:648] HybridClock initialized: now 1786474679957993 us; error 45 us; skew 500 ppm
I20260811 18:57:59.958380 9619 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:57:59 dist-test-slave-h6n0 krb5kdc[8508](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474679, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.112.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:57:59.960757 9619 init.cc:377] Logged in from keytab as oryx/127.29.112.188@KRBTEST.COM (short username oryx)
I20260811 18:57:59.963824 9619 webserver.cc:460] Webserver started at http://127.29.112.188:34927/ using document root <none> and password file <none>
I20260811 18:57:59.964221 9619 fs_manager.cc:359] Metadata directory not provided
I20260811 18:57:59.964351 9619 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:57:59.967682 9619 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.002s
I20260811 18:57:59.969712 10016 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:59.970175 9619 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.002s
I20260811 18:57:59.970335 9619 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/master-2/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/master-2/wal
uuid: "4198b17e42c94aac90009c1bc0970564"
format_stamp: "Formatted at 2026-08-11 18:57:59 on dist-test-slave-h6n0"
server_key: "b425f8b0e97817b19e6deafc76cea9c3"
server_key_iv: "1b4d904cd3036be4cc16858ab601a46a"
server_key_version: "encryptionkey@0"
I20260811 18:57:59.970583 9619 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:58:00.019135 9619 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:58:00.024312 9619 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:58:00.048097 9619 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.188:44651
I20260811 18:58:00.048159 10096 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.188:44651 every 8 connection(s)
I20260811 18:58:00.049409 9619 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/master-2/data/info.pb
I20260811 18:58:00.055481 10097 sys_catalog.cc:263] Verifying existing consensus state
I20260811 18:58:00.059504 10097 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4198b17e42c94aac90009c1bc0970564: Bootstrap starting.
I20260811 18:58:00.076949 10097 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4198b17e42c94aac90009c1bc0970564: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 18:58:00.077502 10097 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4198b17e42c94aac90009c1bc0970564: Bootstrap complete.
I20260811 18:58:00.078729 10097 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4198b17e42c94aac90009c1bc0970564 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b994bc87dbdf4d93a9b50500ad3f2aca" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 37199 } } peers { permanent_uuid: "ea739164f1b344099a538c6c91d0ada6" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 45725 } attrs { promote: false } } peers { permanent_uuid: "4198b17e42c94aac90009c1bc0970564" member_type: NON_VOTER last_known_addr { host: "127.29.112.188" port: 44651 } attrs { promote: true } }
I20260811 18:58:00.079135 10097 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4198b17e42c94aac90009c1bc0970564 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 4198b17e42c94aac90009c1bc0970564, State: Initialized, Role: LEARNER
I20260811 18:58:00.079445 10097 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4198b17e42c94aac90009c1bc0970564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b994bc87dbdf4d93a9b50500ad3f2aca" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 37199 } } peers { permanent_uuid: "ea739164f1b344099a538c6c91d0ada6" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 45725 } attrs { promote: false } } peers { permanent_uuid: "4198b17e42c94aac90009c1bc0970564" member_type: NON_VOTER last_known_addr { host: "127.29.112.188" port: 44651 } attrs { promote: true } }
I20260811 18:58:00.080374 10101 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4198b17e42c94aac90009c1bc0970564 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b994bc87dbdf4d93a9b50500ad3f2aca" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 37199 } } peers { permanent_uuid: "ea739164f1b344099a538c6c91d0ada6" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 45725 } attrs { promote: false } } peers { permanent_uuid: "4198b17e42c94aac90009c1bc0970564" member_type: NON_VOTER last_known_addr { host: "127.29.112.188" port: 44651 } attrs { promote: true } } }
I20260811 18:58:00.080710 10097 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4198b17e42c94aac90009c1bc0970564 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:58:00.080856 10101 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4198b17e42c94aac90009c1bc0970564 [sys.catalog]: This master's current role is: LEARNER
I20260811 18:58:00.095883 10126 catalog_manager.cc:1259] Loaded cluster ID: ddf73710ba7c49fbb2a6a28cae586e9c
I20260811 18:58:00.096024 10126 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 4198b17e42c94aac90009c1bc0970564: loading cluster ID for follower catalog manager: success
I20260811 18:58:00.099622 10126 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 4198b17e42c94aac90009c1bc0970564: acquiring CA information for follower catalog manager: success
I20260811 18:58:00.102023 10126 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 4198b17e42c94aac90009c1bc0970564: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 18:58:00.310495 10127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:00.303610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.188:44651 (local address 127.0.0.1:47620)
0811 18:58:00.304128 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:00.304153 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:00.304290 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:00.304789 (+ 499us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:00.304792 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:00.304812 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:58:00.304993 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:00.305000 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:00.306999 (+ 1999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:00.307003 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:00.308165 (+ 1162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:00.308187 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:00.309919 (+ 1732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:00.309924 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:00.310053 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:00.310079 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 18:58:00.310152 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 18:58:00.310251 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":236,"threads_started":1}
I20260811 18:58:00.310778 10128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:00.303719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47620 (local address 127.29.112.188:44651)
0811 18:58:00.304392 (+ 673us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:00.304399 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:00.304424 (+ 25us) server_negotiation.cc:407] Connection header received
0811 18:58:00.304506 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:00.304513 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:00.304623 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:00.304761 (+ 138us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:58:00.305130 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:00.306840 (+ 1710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:00.308337 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:00.309707 (+ 1370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:00.309950 (+ 243us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:00.310165 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:00.310227 (+ 62us) server_negotiation.cc:299] Negotiation successful
0811 18:58:00.310380 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":547,"thread_start_us":382,"threads_started":1}
I20260811 18:58:00.311682 10052 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 4198b17e42c94aac90009c1bc0970564 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260811 18:58:00.312134 10101 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4198b17e42c94aac90009c1bc0970564 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b994bc87dbdf4d93a9b50500ad3f2aca. Latest consensus state: current_term: 1 leader_uuid: "b994bc87dbdf4d93a9b50500ad3f2aca" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b994bc87dbdf4d93a9b50500ad3f2aca" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 37199 } } peers { permanent_uuid: "ea739164f1b344099a538c6c91d0ada6" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 45725 } attrs { promote: false } } peers { permanent_uuid: "4198b17e42c94aac90009c1bc0970564" member_type: NON_VOTER last_known_addr { host: "127.29.112.188" port: 44651 } attrs { promote: true } } }
I20260811 18:58:00.312460 10101 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4198b17e42c94aac90009c1bc0970564 [sys.catalog]: This master's current role is: LEARNER
W20260811 18:58:00.334249 9993 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:58:00.334511 9993 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:58:00.334673 9993 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:58:00.341939 9993 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:58:00.342037 9993 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:58:00.342252 9993 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.129
I20260811 18:58:00.351823 9993 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:43681
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/krb5kdc/oryx_127.29.112.129.keytab
--principal=oryx/127.29.112.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.129:35525
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.112.129
--webserver_port=37475
--tserver_master_addrs=127.29.112.190:37199,127.29.112.189:45725,127.29.112.188:44651
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.129
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:58:00.353130 9993 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:58:00.354399 9993 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:58:00.367635 10142 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:00.367715 10143 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:00.367760 10146 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:58:00.369221 9993 server_base.cc:1034] running on GCE node
I20260811 18:58:00.369872 9993 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:58:00.370961 9993 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:58:00.372219 9993 hybrid_clock.cc:648] HybridClock initialized: now 1786474680372159 us; error 52 us; skew 500 ppm
I20260811 18:58:00.372573 9993 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:58:00 dist-test-slave-h6n0 krb5kdc[8508](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474680, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.112.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:58:00.376103 9993 init.cc:377] Logged in from keytab as oryx/127.29.112.129@KRBTEST.COM (short username oryx)
I20260811 18:58:00.377803 9993 webserver.cc:460] Webserver started at http://127.29.112.129:37475/ using document root <none> and password file <none>
I20260811 18:58:00.378497 9993 fs_manager.cc:359] Metadata directory not provided
I20260811 18:58:00.378708 9993 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:58:00.384238 9993 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 18:58:00.387087 10156 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:00.388478 9993 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 18:58:00.388615 9993 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/ts-0/wal
uuid: "fc3b99856afe4cc3a86cc67240de20e7"
format_stamp: "Formatted at 2026-08-11 18:57:57 on dist-test-slave-h6n0"
server_key: "9b3521c08e4a877c256cd1ac835ef5ec"
server_key_iv: "f1f480d8f08377da78e668152a0caf33"
server_key_version: "encryptionkey@0"
I20260811 18:58:00.389532 9993 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:58:00.403869 9993 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:58:00.408340 9993 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:58:00.408593 9993 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:58:00.409526 9993 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:58:00.410965 9993 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:58:00.411055 9993 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:00.411132 9993 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:58:00.411190 9993 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:00.448876 9993 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.129:35525
I20260811 18:58:00.448930 10339 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.129:35525 every 8 connection(s)
I20260811 18:58:00.450639 9993 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/ts-0/data/info.pb
I20260811 18:58:00.454610 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 9993
I20260811 18:58:00.454972 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 8961
Aug 11 18:58:00 dist-test-slave-h6n0 krb5kdc[8508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474675, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.29.112.130@KRBTEST.COM
Aug 11 18:58:00 dist-test-slave-h6n0 krb5kdc[8508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474680, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.112.129@KRBTEST.COM for oryx/127.29.112.188@KRBTEST.COM
Aug 11 18:58:00 dist-test-slave-h6n0 krb5kdc[8508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474680, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.112.129@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM
Aug 11 18:58:00 dist-test-slave-h6n0 krb5kdc[8508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474680, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.112.129@KRBTEST.COM for oryx/127.29.112.189@KRBTEST.COM
I20260811 18:58:00.500952 10353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:00.455912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60574 (local address 127.29.112.130:37583)
0811 18:58:00.474420 (+ 18508us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:00.474431 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:00.474468 (+ 37us) server_negotiation.cc:407] Connection header received
0811 18:58:00.474583 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:00.474595 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:00.474856 (+ 261us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:00.476387 (+ 1531us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:00.478125 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:00.479986 (+ 1861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:00.482364 (+ 2378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:00.483415 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:00.488128 (+ 4713us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:00.491295 (+ 3167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:58:00.491314 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:58:00.491343 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:58:00.491386 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:58:00.496118 (+ 4732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:00.497114 (+ 996us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:00.497125 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:00.497129 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:00.497291 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:00.497752 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:00.497757 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:00.497761 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:00.498250 (+ 489us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:58:00.500005 (+ 1755us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:00.500183 (+ 178us) server_negotiation.cc:299] Negotiation successful
0811 18:58:00.500530 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18287,"thread_start_us":19104,"threads_started":1}
I20260811 18:58:00.502280 9196 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:58:00.502485 10349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:00.454802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.129:47087 (local address 127.29.112.190:37199)
0811 18:58:00.458442 (+ 3640us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:00.458452 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:00.465311 (+ 6859us) server_negotiation.cc:407] Connection header received
0811 18:58:00.473190 (+ 7879us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:00.473196 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:00.473287 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:00.473369 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:00.474577 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:00.475668 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:00.482874 (+ 7206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:00.483789 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:00.483920 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:00.492618 (+ 8698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:58:00.492625 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:58:00.492629 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:58:00.492675 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:58:00.497889 (+ 5214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:00.499155 (+ 1266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:00.499161 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:00.499165 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:00.499330 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:00.499908 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:00.499914 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:00.499918 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:00.500376 (+ 458us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:58:00.500522 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:00.501744 (+ 1222us) server_negotiation.cc:299] Negotiation successful
0811 18:58:00.502053 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3470,"thread_start_us":3369,"threads_started":1}
I20260811 18:58:00.504829 10354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:00.454183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37199 (local address 127.29.112.129:47087)
0811 18:58:00.465120 (+ 10937us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:00.465187 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:00.469572 (+ 4385us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:00.473562 (+ 3990us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:00.473569 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:00.474075 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:58:00.474357 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:00.474368 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:00.475859 (+ 1491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:00.475864 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:00.482667 (+ 6803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:00.482684 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:00.484282 (+ 1598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:00.484291 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:00.484731 (+ 440us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:00.485971 (+ 1240us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:58:00.486015 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:58:00.492375 (+ 6360us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:58:00.498057 (+ 5682us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:58:00.498066 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:58:00.498084 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:58:00.499002 (+ 918us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:58:00.499499 (+ 497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:58:00.499505 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:58:00.499509 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:58:00.499763 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:58:00.500638 (+ 875us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:58:00.500643 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:58:00.501158 (+ 515us) client_negotiation.cc:769] Sending connection context
0811 18:58:00.501709 (+ 551us) client_negotiation.cc:240] Negotiation successful
0811 18:58:00.502287 (+ 578us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10618,"mutex_wait_us":22,"spinlock_wait_cycles":28160,"thread_start_us":10340,"threads_started":1}
I20260811 18:58:00.505868 10128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:00.453698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.129:34825 (local address 127.29.112.188:44651)
0811 18:58:00.453928 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:00.453934 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:00.470460 (+ 16526us) server_negotiation.cc:407] Connection header received
0811 18:58:00.470867 (+ 407us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:00.470876 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:00.470989 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:00.471084 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:00.474127 (+ 3043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:00.475299 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:00.480932 (+ 5633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:00.481837 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:00.481979 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:00.492531 (+ 10552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:58:00.492539 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:58:00.492544 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:58:00.492586 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:58:00.496169 (+ 3583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:00.499135 (+ 2966us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:00.499141 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:00.499144 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:00.499343 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:00.499890 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:00.499895 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:00.499897 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:00.500244 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:58:00.500388 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:00.505260 (+ 4872us) server_negotiation.cc:299] Negotiation successful
0811 18:58:00.505504 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 18:58:00.506044 10351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:00.455186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.188:44651 (local address 127.29.112.129:34825)
0811 18:58:00.470295 (+ 15109us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:00.470331 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:00.470693 (+ 362us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:00.471812 (+ 1119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:00.471825 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:00.472947 (+ 1122us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:58:00.473909 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:00.473935 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:00.475526 (+ 1591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:00.475531 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:00.480575 (+ 5044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:00.480599 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:00.482454 (+ 1855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:00.482464 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:00.485421 (+ 2957us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:00.486362 (+ 941us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:58:00.486391 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:58:00.492177 (+ 5786us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:58:00.498069 (+ 5892us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:58:00.498074 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:58:00.498084 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:58:00.498998 (+ 914us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:58:00.499583 (+ 585us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:58:00.499587 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:58:00.499590 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:58:00.499757 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:58:00.500481 (+ 724us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:58:00.500491 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:58:00.500910 (+ 419us) client_negotiation.cc:769] Sending connection context
0811 18:58:00.505133 (+ 4223us) client_negotiation.cc:240] Negotiation successful
0811 18:58:00.505432 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14942,"mutex_wait_us":1837,"spinlock_wait_cycles":113664,"thread_start_us":14577,"threads_started":1}
I20260811 18:58:00.507413 10342 heartbeater.cc:347] Connected to a master server at 127.29.112.188:44651
I20260811 18:58:00.507441 10347 heartbeater.cc:347] Connected to a master server at 127.29.112.190:37199
I20260811 18:58:00.507920 10342 heartbeater.cc:464] Registering TS with master...
I20260811 18:58:00.508013 10347 heartbeater.cc:464] Registering TS with master...
I20260811 18:58:00.509275 10347 heartbeater.cc:511] Master 127.29.112.190:37199 requested a full tablet report, sending...
I20260811 18:58:00.509313 10342 heartbeater.cc:511] Master 127.29.112.188:44651 requested a full tablet report, sending...
I20260811 18:58:00.510267 10356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:00.460181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.129:58537 (local address 127.29.112.189:45725)
0811 18:58:00.485363 (+ 25182us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:00.485372 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:00.486276 (+ 904us) server_negotiation.cc:407] Connection header received
0811 18:58:00.486347 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:00.486353 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:00.486468 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:00.487377 (+ 909us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:00.489457 (+ 2080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:00.490836 (+ 1379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:00.493026 (+ 2190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:00.493736 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:00.493881 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:00.497728 (+ 3847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:58:00.497733 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:58:00.497736 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:58:00.497766 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:58:00.502496 (+ 4730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:00.504151 (+ 1655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:00.504158 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:00.504170 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:00.504479 (+ 309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:00.505675 (+ 1196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:00.505681 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:00.505684 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:00.506889 (+ 1205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:58:00.508919 (+ 2030us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:00.509058 (+ 139us) server_negotiation.cc:299] Negotiation successful
0811 18:58:00.509328 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25022,"thread_start_us":25196,"threads_started":1}
I20260811 18:58:00.511627 8599 ts_manager.cc:194] Re-registered known tserver with Master: fc3b99856afe4cc3a86cc67240de20e7 (127.29.112.129:35525)
I20260811 18:58:00.512509 8599 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.112.129@KRBTEST.COM'} at 127.29.112.129:47087
I20260811 18:58:00.512831 10039 ts_manager.cc:194] Registered new tserver with Master: fc3b99856afe4cc3a86cc67240de20e7 (127.29.112.129:35525)
I20260811 18:58:00.514134 10355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:00.459607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.189:45725 (local address 127.29.112.129:58537)
0811 18:58:00.486006 (+ 26399us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:00.486027 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:00.486164 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:00.487837 (+ 1673us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:00.487844 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:00.488925 (+ 1081us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:58:00.489286 (+ 361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:00.489307 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:00.491043 (+ 1736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:00.491049 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:00.492847 (+ 1798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:00.492865 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:00.494210 (+ 1345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:00.494218 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:00.494427 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:00.494994 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:58:00.495019 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:58:00.497545 (+ 2526us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:58:00.503145 (+ 5600us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:58:00.503154 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:58:00.503158 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:58:00.503716 (+ 558us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:58:00.504691 (+ 975us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:58:00.504699 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:58:00.504704 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:58:00.505487 (+ 783us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:58:00.507064 (+ 1577us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:58:00.507070 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:58:00.507302 (+ 232us) client_negotiation.cc:769] Sending connection context
0811 18:58:00.507481 (+ 179us) client_negotiation.cc:240] Negotiation successful
0811 18:58:00.508356 (+ 875us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26243,"thread_start_us":27119,"threads_started":1}
I20260811 18:58:00.515322 10348 heartbeater.cc:347] Connected to a master server at 127.29.112.189:45725
I20260811 18:58:00.515473 10348 heartbeater.cc:464] Registering TS with master...
I20260811 18:58:00.515895 10348 heartbeater.cc:511] Master 127.29.112.189:45725 requested a full tablet report, sending...
I20260811 18:58:00.517601 9693 ts_manager.cc:194] Registered new tserver with Master: fc3b99856afe4cc3a86cc67240de20e7 (127.29.112.129:35525)
I20260811 18:58:00.600549 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 8961
I20260811 18:58:00.627413 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.130:37583
--local_ip_for_outbound_sockets=127.29.112.130
--tserver_master_addrs=127.29.112.190:37199
--webserver_port=36757
--webserver_interface=127.29.112.130
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/krb5kdc/oryx_127.29.112.130.keytab
--principal=oryx/127.29.112.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:43681
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.29.112.190:37199,127.29.112.189:45725
--tserver_master_addrs=127.29.112.190:37199,127.29.112.189:45725,127.29.112.188:44651 with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 18:58:00.783681 9896 sys_catalog.cc:455] T 00000000000000000000000000000000 P b994bc87dbdf4d93a9b50500ad3f2aca [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b994bc87dbdf4d93a9b50500ad3f2aca" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b994bc87dbdf4d93a9b50500ad3f2aca" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 37199 } } peers { permanent_uuid: "ea739164f1b344099a538c6c91d0ada6" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 45725 } attrs { promote: false } } peers { permanent_uuid: "4198b17e42c94aac90009c1bc0970564" member_type: NON_VOTER last_known_addr { host: "127.29.112.188" port: 44651 } attrs { promote: true } } }
I20260811 18:58:00.784159 9896 sys_catalog.cc:458] T 00000000000000000000000000000000 P b994bc87dbdf4d93a9b50500ad3f2aca [sys.catalog]: This master's current role is: LEADER
I20260811 18:58:00.786496 9896 raft_consensus.cc:1050] T 00000000000000000000000000000000 P b994bc87dbdf4d93a9b50500ad3f2aca: attempting to promote NON_VOTER 4198b17e42c94aac90009c1bc0970564 to VOTER
I20260811 18:58:00.787174 9896 consensus_queue.cc:237] T 00000000000000000000000000000000 P b994bc87dbdf4d93a9b50500ad3f2aca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "b994bc87dbdf4d93a9b50500ad3f2aca" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 37199 } } peers { permanent_uuid: "ea739164f1b344099a538c6c91d0ada6" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 45725 } attrs { promote: false } } peers { permanent_uuid: "4198b17e42c94aac90009c1bc0970564" member_type: VOTER last_known_addr { host: "127.29.112.188" port: 44651 } attrs { promote: false } }
I20260811 18:58:00.789034 9704 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ea739164f1b344099a538c6c91d0ada6 [term 1 FOLLOWER]: Refusing update from remote peer b994bc87dbdf4d93a9b50500ad3f2aca: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260811 18:58:00.789248 10052 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4198b17e42c94aac90009c1bc0970564 [term 1 LEARNER]: Refusing update from remote peer b994bc87dbdf4d93a9b50500ad3f2aca: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260811 18:58:00.789638 10365 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b994bc87dbdf4d93a9b50500ad3f2aca [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea739164f1b344099a538c6c91d0ada6" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 45725 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260811 18:58:00.790148 10364 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b994bc87dbdf4d93a9b50500ad3f2aca [LEADER]: Connected to new peer: Peer: permanent_uuid: "4198b17e42c94aac90009c1bc0970564" member_type: VOTER last_known_addr { host: "127.29.112.188" port: 44651 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260811 18:58:00.793165 9896 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b994bc87dbdf4d93a9b50500ad3f2aca [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 4198b17e42c94aac90009c1bc0970564 (127.29.112.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "b994bc87dbdf4d93a9b50500ad3f2aca" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 37199 } } peers { permanent_uuid: "ea739164f1b344099a538c6c91d0ada6" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 45725 } attrs { promote: false } } peers { permanent_uuid: "4198b17e42c94aac90009c1bc0970564" member_type: VOTER last_known_addr { host: "127.29.112.188" port: 44651 } attrs { promote: false } } }
I20260811 18:58:00.793828 9704 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ea739164f1b344099a538c6c91d0ada6 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 4198b17e42c94aac90009c1bc0970564 (127.29.112.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "b994bc87dbdf4d93a9b50500ad3f2aca" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 37199 } } peers { permanent_uuid: "ea739164f1b344099a538c6c91d0ada6" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 45725 } attrs { promote: false } } peers { permanent_uuid: "4198b17e42c94aac90009c1bc0970564" member_type: VOTER last_known_addr { host: "127.29.112.188" port: 44651 } attrs { promote: false } } }
I20260811 18:58:00.796983 9896 sys_catalog.cc:455] T 00000000000000000000000000000000 P b994bc87dbdf4d93a9b50500ad3f2aca [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "b994bc87dbdf4d93a9b50500ad3f2aca" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "b994bc87dbdf4d93a9b50500ad3f2aca" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 37199 } } peers { permanent_uuid: "ea739164f1b344099a538c6c91d0ada6" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 45725 } attrs { promote: false } } peers { permanent_uuid: "4198b17e42c94aac90009c1bc0970564" member_type: VOTER last_known_addr { host: "127.29.112.188" port: 44651 } attrs { promote: false } } }
I20260811 18:58:00.797302 9896 sys_catalog.cc:458] T 00000000000000000000000000000000 P b994bc87dbdf4d93a9b50500ad3f2aca [sys.catalog]: This master's current role is: LEADER
I20260811 18:58:00.797564 10364 sys_catalog.cc:455] T 00000000000000000000000000000000 P b994bc87dbdf4d93a9b50500ad3f2aca [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b994bc87dbdf4d93a9b50500ad3f2aca" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "b994bc87dbdf4d93a9b50500ad3f2aca" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 37199 } } peers { permanent_uuid: "ea739164f1b344099a538c6c91d0ada6" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 45725 } attrs { promote: false } } peers { permanent_uuid: "4198b17e42c94aac90009c1bc0970564" member_type: VOTER last_known_addr { host: "127.29.112.188" port: 44651 } attrs { promote: false } } }
I20260811 18:58:00.797623 9896 sys_catalog.cc:455] T 00000000000000000000000000000000 P b994bc87dbdf4d93a9b50500ad3f2aca [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b994bc87dbdf4d93a9b50500ad3f2aca" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "b994bc87dbdf4d93a9b50500ad3f2aca" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 37199 } } peers { permanent_uuid: "ea739164f1b344099a538c6c91d0ada6" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 45725 } attrs { promote: false } } peers { permanent_uuid: "4198b17e42c94aac90009c1bc0970564" member_type: VOTER last_known_addr { host: "127.29.112.188" port: 44651 } attrs { promote: false } } }
I20260811 18:58:00.797955 9896 sys_catalog.cc:458] T 00000000000000000000000000000000 P b994bc87dbdf4d93a9b50500ad3f2aca [sys.catalog]: This master's current role is: LEADER
I20260811 18:58:00.798173 10364 sys_catalog.cc:458] T 00000000000000000000000000000000 P b994bc87dbdf4d93a9b50500ad3f2aca [sys.catalog]: This master's current role is: LEADER
I20260811 18:58:00.798012 10052 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 4198b17e42c94aac90009c1bc0970564 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 4198b17e42c94aac90009c1bc0970564 (127.29.112.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "b994bc87dbdf4d93a9b50500ad3f2aca" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 37199 } } peers { permanent_uuid: "ea739164f1b344099a538c6c91d0ada6" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 45725 } attrs { promote: false } } peers { permanent_uuid: "4198b17e42c94aac90009c1bc0970564" member_type: VOTER last_known_addr { host: "127.29.112.188" port: 44651 } attrs { promote: false } } }
I20260811 18:58:00.801265 10101 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4198b17e42c94aac90009c1bc0970564 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b994bc87dbdf4d93a9b50500ad3f2aca" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "b994bc87dbdf4d93a9b50500ad3f2aca" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 37199 } } peers { permanent_uuid: "ea739164f1b344099a538c6c91d0ada6" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 45725 } attrs { promote: false } } peers { permanent_uuid: "4198b17e42c94aac90009c1bc0970564" member_type: VOTER last_known_addr { host: "127.29.112.188" port: 44651 } attrs { promote: false } } }
I20260811 18:58:00.801563 10101 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4198b17e42c94aac90009c1bc0970564 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:58:00.805173 10368 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea739164f1b344099a538c6c91d0ada6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b994bc87dbdf4d93a9b50500ad3f2aca" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "b994bc87dbdf4d93a9b50500ad3f2aca" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 37199 } } peers { permanent_uuid: "ea739164f1b344099a538c6c91d0ada6" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 45725 } attrs { promote: false } } peers { permanent_uuid: "4198b17e42c94aac90009c1bc0970564" member_type: VOTER last_known_addr { host: "127.29.112.188" port: 44651 } attrs { promote: false } } }
I20260811 18:58:00.805575 10368 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea739164f1b344099a538c6c91d0ada6 [sys.catalog]: This master's current role is: FOLLOWER
W20260811 18:58:00.972985 10363 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:58:00.973242 10363 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:58:00.973337 10363 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:58:00.981966 10363 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:58:00.982069 10363 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:58:00.982239 10363 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.130
I20260811 18:58:00.992095 10363 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:43681
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/krb5kdc/oryx_127.29.112.130.keytab
--principal=oryx/127.29.112.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.130:37583
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.112.130
--webserver_port=36757
--tserver_master_addrs=127.29.112.190:37199,127.29.112.189:45725,127.29.112.188:44651
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.130
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:58:00.993321 10363 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:58:00.994750 10363 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:58:01.008134 10379 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:01.008183 10380 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:01.008697 10384 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:58:01.009703 10363 server_base.cc:1034] running on GCE node
I20260811 18:58:01.010385 10363 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:58:01.011447 10363 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:58:01.012645 10363 hybrid_clock.cc:648] HybridClock initialized: now 1786474681012591 us; error 48 us; skew 500 ppm
I20260811 18:58:01.013012 10363 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:58:01 dist-test-slave-h6n0 krb5kdc[8508](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474681, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.112.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:58:01.016228 10363 init.cc:377] Logged in from keytab as oryx/127.29.112.130@KRBTEST.COM (short username oryx)
I20260811 18:58:01.017652 10363 webserver.cc:460] Webserver started at http://127.29.112.130:36757/ using document root <none> and password file <none>
I20260811 18:58:01.018317 10363 fs_manager.cc:359] Metadata directory not provided
I20260811 18:58:01.018496 10363 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:58:01.023696 10363 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.004s
I20260811 18:58:01.026679 10393 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:01.028185 10363 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 18:58:01.028407 10363 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/ts-1/wal
uuid: "aaab81ed038149258391f548b07268af"
format_stamp: "Formatted at 2026-08-11 18:57:57 on dist-test-slave-h6n0"
server_key: "da63a013e79d4a56ee27953c95881e92"
server_key_iv: "411c0352d123579b1fe7843627aaf21a"
server_key_version: "encryptionkey@0"
I20260811 18:58:01.029356 10363 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:58:01.048822 10363 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:58:01.053602 10363 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:58:01.053900 10363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:58:01.054991 10363 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:58:01.056479 10363 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:58:01.056567 10363 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:01.056710 10363 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:58:01.056789 10363 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:01.101972 10363 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.130:37583
I20260811 18:58:01.102056 10589 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.130:37583 every 8 connection(s)
I20260811 18:58:01.103868 10363 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/ts-1/data/info.pb
I20260811 18:58:01.113647 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 10363
I20260811 18:58:01.113979 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 9216
Aug 11 18:58:01 dist-test-slave-h6n0 krb5kdc[8508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474681, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.112.130@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM
Aug 11 18:58:01 dist-test-slave-h6n0 krb5kdc[8508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474681, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.112.130@KRBTEST.COM for oryx/127.29.112.189@KRBTEST.COM
Aug 11 18:58:01 dist-test-slave-h6n0 krb5kdc[8508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474681, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.112.130@KRBTEST.COM for oryx/127.29.112.188@KRBTEST.COM
Aug 11 18:58:01 dist-test-slave-h6n0 krb5kdc[8508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474675, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.29.112.131@KRBTEST.COM
I20260811 18:58:01.148434 10600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:01.112128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.130:55115 (local address 127.29.112.190:37199)
0811 18:58:01.112713 (+ 585us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:01.112719 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:01.112742 (+ 23us) server_negotiation.cc:407] Connection header received
0811 18:58:01.122119 (+ 9377us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:01.122127 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:01.122231 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:01.122760 (+ 529us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:01.125196 (+ 2436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:01.125918 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:01.128662 (+ 2744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:01.129256 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:01.129422 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:01.134967 (+ 5545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:58:01.134975 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:58:01.134980 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:58:01.135026 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:58:01.138645 (+ 3619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:01.140048 (+ 1403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:01.140053 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:01.140057 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:01.140224 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:01.144697 (+ 4473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:01.144703 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:01.144707 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:01.145109 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:58:01.145284 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:01.147560 (+ 2276us) server_negotiation.cc:299] Negotiation successful
0811 18:58:01.147883 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":438,"thread_start_us":935,"threads_started":1}
I20260811 18:58:01.152962 10601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:01.112590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.189:45725 (local address 127.29.112.130:60947)
0811 18:58:01.114807 (+ 2217us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:01.114831 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:01.121793 (+ 6962us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:01.123557 (+ 1764us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:01.123561 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:01.124315 (+ 754us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:58:01.125056 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:01.125078 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:01.126982 (+ 1904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:01.126988 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:01.128809 (+ 1821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:01.128828 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:01.130164 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:01.130170 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:01.130562 (+ 392us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:01.137388 (+ 6826us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:58:01.137416 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:58:01.143347 (+ 5931us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:58:01.149436 (+ 6089us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:58:01.149443 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:58:01.149447 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:58:01.150128 (+ 681us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:58:01.150660 (+ 532us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:58:01.150665 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:58:01.150668 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:58:01.150851 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:58:01.151621 (+ 770us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:58:01.151629 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:58:01.151812 (+ 183us) client_negotiation.cc:769] Sending connection context
0811 18:58:01.151992 (+ 180us) client_negotiation.cc:240] Negotiation successful
0811 18:58:01.152349 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2049,"mutex_wait_us":145,"thread_start_us":391,"threads_started":1}
I20260811 18:58:01.153987 10596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:01.110997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.130:60947 (local address 127.29.112.189:45725)
0811 18:58:01.111528 (+ 531us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:01.111536 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:01.114940 (+ 3404us) server_negotiation.cc:407] Connection header received
0811 18:58:01.123135 (+ 8195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:01.123144 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:01.123258 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:01.123354 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:01.125268 (+ 1914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:01.126711 (+ 1443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:01.129009 (+ 2298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:01.129738 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:01.129844 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:01.143669 (+ 13825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:58:01.143678 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:58:01.143684 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:58:01.143745 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:58:01.149290 (+ 5545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:01.150289 (+ 999us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:01.150295 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:01.150299 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:01.150471 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:01.150989 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:01.150993 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:01.150996 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:01.151390 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:58:01.151541 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:01.153329 (+ 1788us) server_negotiation.cc:299] Negotiation successful
0811 18:58:01.153582 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":521,"threads_started":1}
I20260811 18:58:01.155645 10598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:01.111338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.130:49917 (local address 127.29.112.188:44651)
0811 18:58:01.111926 (+ 588us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:01.111933 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:01.131155 (+ 19222us) server_negotiation.cc:407] Connection header received
0811 18:58:01.131317 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:01.131322 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:01.131445 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:01.131585 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:01.132761 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:01.134018 (+ 1257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:01.137350 (+ 3332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:01.138029 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:01.138139 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:01.146394 (+ 8255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:58:01.146403 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:58:01.146410 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:58:01.146467 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:58:01.152042 (+ 5575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:01.153055 (+ 1013us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:01.153062 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:01.153066 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:01.153205 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:01.153706 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:01.153714 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:01.153719 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:01.154130 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:58:01.154262 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:01.154884 (+ 622us) server_negotiation.cc:299] Negotiation successful
0811 18:58:01.155153 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":413,"thread_start_us":272,"threads_started":1}
I20260811 18:58:01.156044 10593 heartbeater.cc:347] Connected to a master server at 127.29.112.189:45725
I20260811 18:58:01.156399 10593 heartbeater.cc:464] Registering TS with master...
I20260811 18:58:01.156776 10602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:01.112814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.188:44651 (local address 127.29.112.130:49917)
0811 18:58:01.131003 (+ 18189us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:01.131024 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:01.131168 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:01.131600 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:01.131605 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:01.132169 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:58:01.132571 (+ 402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:01.132587 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:01.134180 (+ 1593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:01.134185 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:01.137158 (+ 2973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:01.137175 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:01.138497 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:01.138503 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:01.138704 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:01.139763 (+ 1059us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:58:01.139785 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:58:01.146077 (+ 6292us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:58:01.152227 (+ 6150us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:58:01.152235 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:58:01.152240 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:58:01.152884 (+ 644us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:58:01.153362 (+ 478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:58:01.153368 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:58:01.153371 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:58:01.153557 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:58:01.154350 (+ 793us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:58:01.154357 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:58:01.154551 (+ 194us) client_negotiation.cc:769] Sending connection context
0811 18:58:01.155857 (+ 1306us) client_negotiation.cc:240] Negotiation successful
0811 18:58:01.156211 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18037,"thread_start_us":18441,"threads_started":1}
I20260811 18:58:01.157411 10593 heartbeater.cc:511] Master 127.29.112.189:45725 requested a full tablet report, sending...
I20260811 18:58:01.157596 10591 heartbeater.cc:347] Connected to a master server at 127.29.112.188:44651
I20260811 18:58:01.157727 10591 heartbeater.cc:464] Registering TS with master...
I20260811 18:58:01.158243 10591 heartbeater.cc:511] Master 127.29.112.188:44651 requested a full tablet report, sending...
I20260811 18:58:01.158892 10599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:01.111488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37199 (local address 127.29.112.130:55115)
0811 18:58:01.112435 (+ 947us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:01.112486 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:01.121552 (+ 9066us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:01.123025 (+ 1473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:01.123037 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:01.123916 (+ 879us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:58:01.125012 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:01.125037 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:01.126105 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:01.126110 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:01.128490 (+ 2380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:01.128510 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:01.129529 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:01.129537 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:01.129922 (+ 385us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:01.131024 (+ 1102us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:58:01.131061 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:58:01.134718 (+ 3657us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:58:01.138808 (+ 4090us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:58:01.138819 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:58:01.138837 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:58:01.139552 (+ 715us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:58:01.141222 (+ 1670us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:58:01.141226 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:58:01.141229 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:58:01.141479 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:58:01.145480 (+ 4001us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:58:01.145503 (+ 23us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:58:01.146436 (+ 933us) client_negotiation.cc:769] Sending connection context
0811 18:58:01.146968 (+ 532us) client_negotiation.cc:240] Negotiation successful
0811 18:58:01.147843 (+ 875us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":589,"mutex_wait_us":47,"thread_start_us":1503,"threads_started":1}
I20260811 18:58:01.159725 9693 ts_manager.cc:194] Registered new tserver with Master: aaab81ed038149258391f548b07268af (127.29.112.130:37583)
I20260811 18:58:01.159844 10039 ts_manager.cc:194] Registered new tserver with Master: aaab81ed038149258391f548b07268af (127.29.112.130:37583)
I20260811 18:58:01.160506 10607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:01.115588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53270 (local address 127.29.112.131:41101)
0811 18:58:01.116346 (+ 758us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:01.116357 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:01.122271 (+ 5914us) server_negotiation.cc:407] Connection header received
0811 18:58:01.123420 (+ 1149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:01.123445 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:01.123710 (+ 265us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:01.123856 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:01.133299 (+ 9443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:01.134754 (+ 1455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:01.137968 (+ 3214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:01.138917 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:01.139662 (+ 745us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:01.150966 (+ 11304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:58:01.150986 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:58:01.151002 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:58:01.151059 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:58:01.155183 (+ 4124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:01.156106 (+ 923us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:01.156114 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:01.156119 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:01.156285 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:01.156787 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:01.156792 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:01.156795 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:01.157215 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:58:01.157534 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:01.158258 (+ 724us) server_negotiation.cc:299] Negotiation successful
0811 18:58:01.158697 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":541,"thread_start_us":381,"threads_started":1}
I20260811 18:58:01.160601 10592 heartbeater.cc:347] Connected to a master server at 127.29.112.190:37199
I20260811 18:58:01.160728 10592 heartbeater.cc:464] Registering TS with master...
I20260811 18:58:01.161193 10592 heartbeater.cc:511] Master 127.29.112.190:37199 requested a full tablet report, sending...
I20260811 18:58:01.162107 8599 ts_manager.cc:194] Re-registered known tserver with Master: aaab81ed038149258391f548b07268af (127.29.112.130:37583)
I20260811 18:58:01.162179 9433 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:58:01.163511 8599 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.112.130@KRBTEST.COM'} at 127.29.112.130:55115
I20260811 18:58:01.249276 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 9216
I20260811 18:58:01.274276 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.131:41101
--local_ip_for_outbound_sockets=127.29.112.131
--tserver_master_addrs=127.29.112.190:37199
--webserver_port=33049
--webserver_interface=127.29.112.131
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/krb5kdc/oryx_127.29.112.131.keytab
--principal=oryx/127.29.112.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:43681
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.29.112.190:37199,127.29.112.189:45725
--tserver_master_addrs=127.29.112.190:37199,127.29.112.189:45725,127.29.112.188:44651 with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 18:58:01.515195 10347 heartbeater.cc:503] Master 127.29.112.190:37199 was elected leader, sending a full tablet report...
W20260811 18:58:01.625337 10611 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:58:01.625638 10611 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:58:01.625734 10611 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:58:01.633661 10611 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:58:01.633898 10611 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:58:01.634060 10611 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.131
I20260811 18:58:01.644840 10611 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:43681
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/krb5kdc/oryx_127.29.112.131.keytab
--principal=oryx/127.29.112.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.131:41101
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.112.131
--webserver_port=33049
--tserver_master_addrs=127.29.112.190:37199,127.29.112.189:45725,127.29.112.188:44651
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.131
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:58:01.646167 10611 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:58:01.647519 10611 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:58:01.660610 10622 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:01.660624 10623 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:01.660761 10627 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:58:01.661993 10611 server_base.cc:1034] running on GCE node
I20260811 18:58:01.662707 10611 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:58:01.663652 10611 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:58:01.664865 10611 hybrid_clock.cc:648] HybridClock initialized: now 1786474681664828 us; error 31 us; skew 500 ppm
I20260811 18:58:01.665241 10611 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:58:01 dist-test-slave-h6n0 krb5kdc[8508](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474681, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.112.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:58:01.678969 10611 init.cc:377] Logged in from keytab as oryx/127.29.112.131@KRBTEST.COM (short username oryx)
I20260811 18:58:01.681221 10611 webserver.cc:460] Webserver started at http://127.29.112.131:33049/ using document root <none> and password file <none>
I20260811 18:58:01.682118 10611 fs_manager.cc:359] Metadata directory not provided
I20260811 18:58:01.682389 10611 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:58:01.688673 10611 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 18:58:01.691792 10640 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:01.693332 10611 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 18:58:01.693549 10611 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/ts-2/wal
uuid: "fa32c49cd161420c979c48cd9fad3b05"
format_stamp: "Formatted at 2026-08-11 18:57:58 on dist-test-slave-h6n0"
server_key: "9a678c2cb558f7b88b9c11b75f99ef21"
server_key_iv: "47a019c22dcd7bd0696ca97b80f1ecb3"
server_key_version: "encryptionkey@0"
I20260811 18:58:01.694767 10611 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:58:01.715258 10611 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:58:01.720168 10611 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:58:01.720500 10611 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:58:01.721525 10611 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:58:01.723088 10611 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:58:01.723184 10611 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:01.723317 10611 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:58:01.723415 10611 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:01.762043 10611 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.131:41101
I20260811 18:58:01.762248 10830 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.131:41101 every 8 connection(s)
I20260811 18:58:01.763535 10611 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/ts-2/data/info.pb
I20260811 18:58:01.771737 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 10611
Aug 11 18:58:01 dist-test-slave-h6n0 krb5kdc[8508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474681, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.112.131@KRBTEST.COM for oryx/127.29.112.188@KRBTEST.COM
I20260811 18:58:01.819314 10845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:01.766469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.131:57031 (local address 127.29.112.188:44651)
0811 18:58:01.770091 (+ 3622us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:01.770100 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:01.788083 (+ 17983us) server_negotiation.cc:407] Connection header received
0811 18:58:01.790117 (+ 2034us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:01.790123 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:01.790231 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:01.790353 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:01.792796 (+ 2443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:01.794008 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:01.796606 (+ 2598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:01.797352 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:01.797496 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:01.807438 (+ 9942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:58:01.807448 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:58:01.807456 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:58:01.807512 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:58:01.811672 (+ 4160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:01.812774 (+ 1102us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:01.812779 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:01.812783 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:01.812934 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:01.814053 (+ 1119us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:01.814058 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:01.814063 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:01.814520 (+ 457us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:58:01.814717 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:01.818697 (+ 3980us) server_negotiation.cc:299] Negotiation successful
0811 18:58:01.818926 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3418,"thread_start_us":3276,"threads_started":1}
I20260811 18:58:01.818507 10852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:01.771371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.188:44651 (local address 127.29.112.131:57031)
0811 18:58:01.787902 (+ 16531us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:01.787943 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:01.789831 (+ 1888us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:01.790863 (+ 1032us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:01.790876 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:01.791775 (+ 899us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:58:01.792627 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:01.792646 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:01.794224 (+ 1578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:01.794229 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:01.796357 (+ 2128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:01.796378 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:01.797902 (+ 1524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:01.797913 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:01.799810 (+ 1897us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:01.800730 (+ 920us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:58:01.800768 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:58:01.805865 (+ 5097us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:58:01.811837 (+ 5972us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:58:01.811846 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:58:01.811865 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:58:01.812622 (+ 757us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:58:01.813630 (+ 1008us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:58:01.813635 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:58:01.813639 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:58:01.813907 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:58:01.815227 (+ 1320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:58:01.815238 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:58:01.815778 (+ 540us) client_negotiation.cc:769] Sending connection context
0811 18:58:01.816279 (+ 501us) client_negotiation.cc:240] Negotiation successful
0811 18:58:01.816923 (+ 644us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16245,"mutex_wait_us":1427,"thread_start_us":21487,"threads_started":1}
I20260811 18:58:01.824795 10833 heartbeater.cc:347] Connected to a master server at 127.29.112.188:44651
I20260811 18:58:01.825202 10833 heartbeater.cc:464] Registering TS with master...
I20260811 18:58:01.826359 10833 heartbeater.cc:511] Master 127.29.112.188:44651 requested a full tablet report, sending...
Aug 11 18:58:01 dist-test-slave-h6n0 krb5kdc[8508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474681, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.112.131@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM
I20260811 18:58:01.828421 10039 ts_manager.cc:194] Registered new tserver with Master: fa32c49cd161420c979c48cd9fad3b05 (127.29.112.131:41101)
Aug 11 18:58:01 dist-test-slave-h6n0 krb5kdc[8508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474681, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.112.131@KRBTEST.COM for oryx/127.29.112.189@KRBTEST.COM
I20260811 18:58:01.838456 10851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:01.772166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37199 (local address 127.29.112.131:35655)
0811 18:58:01.790714 (+ 18548us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:01.790739 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:01.790911 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:01.791504 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:01.791508 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:01.793569 (+ 2061us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:58:01.793946 (+ 377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:01.793963 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:01.796593 (+ 2630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:01.796602 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:01.822061 (+ 25459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:01.822082 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:01.823423 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:01.823433 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:01.823696 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:01.824399 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:58:01.824432 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:58:01.830182 (+ 5750us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:58:01.835040 (+ 4858us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:58:01.835050 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:58:01.835057 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:58:01.835957 (+ 900us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:58:01.836473 (+ 516us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:58:01.836481 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:58:01.836486 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:58:01.836654 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:58:01.837270 (+ 616us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:58:01.837277 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:58:01.837465 (+ 188us) client_negotiation.cc:769] Sending connection context
0811 18:58:01.837623 (+ 158us) client_negotiation.cc:240] Negotiation successful
0811 18:58:01.837925 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18398,"mutex_wait_us":21487,"thread_start_us":18121,"threads_started":1}
I20260811 18:58:01.838460 10849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:01.766506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.131:35655 (local address 127.29.112.190:37199)
0811 18:58:01.780036 (+ 13530us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:01.780045 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:01.790820 (+ 10775us) server_negotiation.cc:407] Connection header received
0811 18:58:01.791067 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:01.791073 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:01.791159 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:01.791247 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:01.794100 (+ 2853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:01.795275 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:01.822265 (+ 26990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:01.823197 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:01.823334 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:01.830578 (+ 7244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:58:01.830588 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:58:01.830594 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:58:01.830652 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:58:01.834786 (+ 4134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:01.836129 (+ 1343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:01.836139 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:01.836146 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:01.836326 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:01.836763 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:01.836768 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:01.836772 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:01.837122 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:58:01.837269 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:01.837745 (+ 476us) server_negotiation.cc:299] Negotiation successful
0811 18:58:01.837991 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13346,"thread_start_us":13093,"threads_started":1}
I20260811 18:58:01.839884 10834 heartbeater.cc:347] Connected to a master server at 127.29.112.190:37199
I20260811 18:58:01.840049 10834 heartbeater.cc:464] Registering TS with master...
I20260811 18:58:01.840585 10834 heartbeater.cc:511] Master 127.29.112.190:37199 requested a full tablet report, sending...
I20260811 18:58:01.841718 8599 ts_manager.cc:194] Re-registered known tserver with Master: fa32c49cd161420c979c48cd9fad3b05 (127.29.112.131:41101)
I20260811 18:58:01.842944 8599 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.112.131@KRBTEST.COM'} at 127.29.112.131:35655
I20260811 18:58:01.845229 10848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:01.766469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.131:39717 (local address 127.29.112.189:45725)
0811 18:58:01.779598 (+ 13129us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:01.779608 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:01.793393 (+ 13785us) server_negotiation.cc:407] Connection header received
0811 18:58:01.793618 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:01.793624 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:01.793715 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:01.793810 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:01.810752 (+ 16942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:01.811957 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:01.818491 (+ 6534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:01.819203 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:01.819313 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:01.836316 (+ 17003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:58:01.836322 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:58:01.836326 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:58:01.836369 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:58:01.840438 (+ 4069us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:01.842104 (+ 1666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:01.842113 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:01.842117 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:01.842328 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:01.842914 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:01.842921 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:01.842925 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:01.843473 (+ 548us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:58:01.843670 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:01.844381 (+ 711us) server_negotiation.cc:299] Negotiation successful
0811 18:58:01.844693 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12954,"thread_start_us":12828,"threads_started":1}
I20260811 18:58:01.846238 10850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:01.772701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.189:45725 (local address 127.29.112.131:39717)
0811 18:58:01.793288 (+ 20587us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:01.793306 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:01.793481 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:01.806121 (+ 12640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:01.806128 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:01.806880 (+ 752us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:58:01.807298 (+ 418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:01.807313 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:01.812261 (+ 4948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:01.812266 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:01.814013 (+ 1747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:01.814027 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:01.819872 (+ 5845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:01.819882 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:01.820112 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:01.830652 (+ 10540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:58:01.830683 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:58:01.834326 (+ 3643us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:58:01.840710 (+ 6384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:58:01.840717 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:58:01.840721 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:58:01.841914 (+ 1193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:58:01.842507 (+ 593us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:58:01.842515 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:58:01.842521 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:58:01.842740 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:58:01.843826 (+ 1086us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:58:01.843832 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:58:01.844024 (+ 192us) client_negotiation.cc:769] Sending connection context
0811 18:58:01.845455 (+ 1431us) client_negotiation.cc:240] Negotiation successful
0811 18:58:01.845821 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20462,"thread_start_us":14791,"threads_started":1}
I20260811 18:58:01.847244 10838 heartbeater.cc:347] Connected to a master server at 127.29.112.189:45725
I20260811 18:58:01.847383 10838 heartbeater.cc:464] Registering TS with master...
I20260811 18:58:01.847807 10838 heartbeater.cc:511] Master 127.29.112.189:45725 requested a full tablet report, sending...
I20260811 18:58:01.849172 9693 ts_manager.cc:194] Registered new tserver with Master: fa32c49cd161420c979c48cd9fad3b05 (127.29.112.131:41101)
I20260811 18:58:01.851161 10859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:01.793955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56282 (local address 127.29.112.189:45725)
0811 18:58:01.820944 (+ 26989us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:01.820953 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:01.820978 (+ 25us) server_negotiation.cc:407] Connection header received
0811 18:58:01.821040 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:01.821048 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:01.821143 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:01.821271 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:01.829539 (+ 8268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:01.831753 (+ 2214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:01.834829 (+ 3076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:01.835281 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:01.835423 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:01.843748 (+ 8325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:58:01.843756 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:58:01.843761 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:58:01.843799 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:58:01.847605 (+ 3806us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:01.848574 (+ 969us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:01.848579 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:01.848582 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:01.848759 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:01.849213 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:01.849218 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:01.849221 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:01.849583 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:58:01.849820 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:01.850484 (+ 664us) server_negotiation.cc:299] Negotiation successful
0811 18:58:01.850748 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26830,"thread_start_us":26722,"threads_started":1}
I20260811 18:58:01.854851 10861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:01.795761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47630 (local address 127.29.112.188:44651)
0811 18:58:01.818289 (+ 22528us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:01.818294 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:01.824284 (+ 5990us) server_negotiation.cc:407] Connection header received
0811 18:58:01.824351 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:01.824356 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:01.824425 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:01.824519 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:01.829369 (+ 4850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:01.830589 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:01.838781 (+ 8192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:01.840076 (+ 1295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:01.840206 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:01.846227 (+ 6021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:58:01.846236 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:58:01.846241 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:58:01.846557 (+ 316us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:58:01.851769 (+ 5212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:01.852679 (+ 910us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:01.852684 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:01.852688 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:01.852844 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:01.853270 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:01.853273 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:01.853276 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:01.853545 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:58:01.853828 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:01.854159 (+ 331us) server_negotiation.cc:299] Negotiation successful
0811 18:58:01.854388 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22405,"thread_start_us":22548,"threads_started":1}
I20260811 18:58:01.860234 10862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:01.798355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60544 (local address 127.29.112.190:37199)
0811 18:58:01.826752 (+ 28397us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:01.826760 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:01.836587 (+ 9827us) server_negotiation.cc:407] Connection header received
0811 18:58:01.837878 (+ 1291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:01.837882 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:01.837957 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:01.838036 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:01.842002 (+ 3966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:01.843128 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:01.847810 (+ 4682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:01.848552 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:01.848683 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:01.853544 (+ 4861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:58:01.853550 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:58:01.853553 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:58:01.853586 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:58:01.857338 (+ 3752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:01.858052 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:01.858059 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:01.858063 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:01.858205 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:01.858668 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:01.858674 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:01.858683 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:01.858971 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:58:01.859257 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:01.859627 (+ 370us) server_negotiation.cc:299] Negotiation successful
0811 18:58:01.859878 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28215,"thread_start_us":29949,"threads_started":1}
I20260811 18:58:01.864737 8599 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60544:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:58:01.871999 8599 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:58:01.906852 10886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:01.894820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60580 (local address 127.29.112.130:37583)
0811 18:58:01.896677 (+ 1857us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:01.896694 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:01.896753 (+ 59us) server_negotiation.cc:407] Connection header received
0811 18:58:01.896936 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:01.896952 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:01.897455 (+ 503us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:01.897977 (+ 522us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:58:01.898749 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:01.900201 (+ 1452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:01.903813 (+ 3612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:01.905486 (+ 1673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:01.905673 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:01.905823 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:01.906383 (+ 560us) server_negotiation.cc:299] Negotiation successful
0811 18:58:01.906602 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1021,"thread_start_us":1265,"threads_started":1}
I20260811 18:58:01.906913 10885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:01.894268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.130:37583 (local address 127.0.0.1:60580)
0811 18:58:01.895121 (+ 853us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:01.895143 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:01.895278 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:01.897906 (+ 2628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:01.897914 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:01.897954 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:58:01.898435 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:01.898452 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:01.900469 (+ 2017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:01.900478 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:01.903528 (+ 3050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:01.903555 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:01.905987 (+ 2432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:01.905992 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:01.906166 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:01.906205 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 18:58:01.906307 (+ 102us) client_negotiation.cc:240] Negotiation successful
0811 18:58:01.906443 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":663,"mutex_wait_us":27,"thread_start_us":4476,"threads_started":1}
I20260811 18:58:01.907563 10881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:01.893456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:35525 (local address 127.0.0.1:35810)
0811 18:58:01.894213 (+ 757us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:01.894234 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:01.894356 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:01.900521 (+ 6165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:01.900524 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:01.900549 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:58:01.900988 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:01.901003 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:01.902853 (+ 1850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:01.902858 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:01.904803 (+ 1945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:01.904816 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:01.906756 (+ 1940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:01.906762 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:01.906967 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:01.907000 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 18:58:01.907083 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 18:58:01.907209 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":598,"mutex_wait_us":19,"thread_start_us":449,"threads_started":1}
I20260811 18:58:01.908041 10883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:01.893674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35810 (local address 127.29.112.129:35525)
0811 18:58:01.899472 (+ 5798us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:01.899483 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:01.899516 (+ 33us) server_negotiation.cc:407] Connection header received
0811 18:58:01.899691 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:01.899699 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:01.900099 (+ 400us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:01.900559 (+ 460us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:58:01.901201 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:01.902666 (+ 1465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:01.904966 (+ 2300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:01.906499 (+ 1533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:01.906973 (+ 474us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:01.907138 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:01.907586 (+ 448us) server_negotiation.cc:299] Negotiation successful
0811 18:58:01.907758 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5611,"thread_start_us":4801,"threads_started":1}
I20260811 18:58:01.911250 10229 tablet_service.cc:1467] Processing CreateTablet for tablet 08e01fa3f73e49a7ba32fc8e84681c0b (DEFAULT_TABLE table=test-table [id=cc0bb03df40c44fbab8454dc4b84ad64]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:58:01.911252 10480 tablet_service.cc:1467] Processing CreateTablet for tablet 08e01fa3f73e49a7ba32fc8e84681c0b (DEFAULT_TABLE table=test-table [id=cc0bb03df40c44fbab8454dc4b84ad64]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:58:01.912611 10229 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08e01fa3f73e49a7ba32fc8e84681c0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:58:01.913542 10480 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08e01fa3f73e49a7ba32fc8e84681c0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:58:01.920224 10887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:01.895526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53278 (local address 127.29.112.131:41101)
0811 18:58:01.896636 (+ 1110us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:01.896649 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:01.899251 (+ 2602us) server_negotiation.cc:407] Connection header received
0811 18:58:01.901048 (+ 1797us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:01.901056 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:01.901262 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:01.901458 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:58:01.902586 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:01.904888 (+ 2302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:01.908389 (+ 3501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:01.909894 (+ 1505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:01.910055 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:01.910204 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:01.919416 (+ 9212us) server_negotiation.cc:299] Negotiation successful
0811 18:58:01.919628 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":853,"thread_start_us":756,"threads_started":1}
I20260811 18:58:01.920311 10884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:01.894344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.131:41101 (local address 127.0.0.1:53278)
0811 18:58:01.899047 (+ 4703us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:01.899090 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:01.900844 (+ 1754us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:01.901455 (+ 611us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:01.901459 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:01.901481 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:58:01.902141 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:01.902168 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:01.905108 (+ 2940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:01.905113 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:01.908231 (+ 3118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:01.908246 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:01.918794 (+ 10548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:01.918802 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:01.918997 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:01.919031 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 18:58:01.919196 (+ 165us) client_negotiation.cc:240] Negotiation successful
0811 18:58:01.919518 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4579,"mutex_wait_us":5,"thread_start_us":386,"threads_started":1}
I20260811 18:58:01.924165 10719 tablet_service.cc:1467] Processing CreateTablet for tablet 08e01fa3f73e49a7ba32fc8e84681c0b (DEFAULT_TABLE table=test-table [id=cc0bb03df40c44fbab8454dc4b84ad64]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:58:01.925662 10719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08e01fa3f73e49a7ba32fc8e84681c0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:58:01.926728 10888 tablet_bootstrap.cc:492] T 08e01fa3f73e49a7ba32fc8e84681c0b P fc3b99856afe4cc3a86cc67240de20e7: Bootstrap starting.
I20260811 18:58:01.929461 10890 tablet_bootstrap.cc:492] T 08e01fa3f73e49a7ba32fc8e84681c0b P aaab81ed038149258391f548b07268af: Bootstrap starting.
I20260811 18:58:01.930383 10888 tablet_bootstrap.cc:654] T 08e01fa3f73e49a7ba32fc8e84681c0b P fc3b99856afe4cc3a86cc67240de20e7: Neither blocks nor log segments found. Creating new log.
I20260811 18:58:01.931627 10888 log.cc:824] T 08e01fa3f73e49a7ba32fc8e84681c0b P fc3b99856afe4cc3a86cc67240de20e7: Log is configured to *not* fsync() on all Append() calls
I20260811 18:58:01.932803 10890 tablet_bootstrap.cc:654] T 08e01fa3f73e49a7ba32fc8e84681c0b P aaab81ed038149258391f548b07268af: Neither blocks nor log segments found. Creating new log.
I20260811 18:58:01.934005 10890 log.cc:824] T 08e01fa3f73e49a7ba32fc8e84681c0b P aaab81ed038149258391f548b07268af: Log is configured to *not* fsync() on all Append() calls
I20260811 18:58:01.934535 10888 tablet_bootstrap.cc:492] T 08e01fa3f73e49a7ba32fc8e84681c0b P fc3b99856afe4cc3a86cc67240de20e7: No bootstrap required, opened a new log
I20260811 18:58:01.934856 10888 ts_tablet_manager.cc:1397] T 08e01fa3f73e49a7ba32fc8e84681c0b P fc3b99856afe4cc3a86cc67240de20e7: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.006s
I20260811 18:58:01.937387 10890 tablet_bootstrap.cc:492] T 08e01fa3f73e49a7ba32fc8e84681c0b P aaab81ed038149258391f548b07268af: No bootstrap required, opened a new log
I20260811 18:58:01.937716 10890 ts_tablet_manager.cc:1397] T 08e01fa3f73e49a7ba32fc8e84681c0b P aaab81ed038149258391f548b07268af: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260811 18:58:01.942010 10894 tablet_bootstrap.cc:492] T 08e01fa3f73e49a7ba32fc8e84681c0b P fa32c49cd161420c979c48cd9fad3b05: Bootstrap starting.
I20260811 18:58:01.943342 10888 raft_consensus.cc:348] T 08e01fa3f73e49a7ba32fc8e84681c0b P fc3b99856afe4cc3a86cc67240de20e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc3b99856afe4cc3a86cc67240de20e7" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 35525 } } peers { permanent_uuid: "aaab81ed038149258391f548b07268af" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 37583 } } peers { permanent_uuid: "fa32c49cd161420c979c48cd9fad3b05" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 41101 } }
I20260811 18:58:01.943789 10888 raft_consensus.cc:374] T 08e01fa3f73e49a7ba32fc8e84681c0b P fc3b99856afe4cc3a86cc67240de20e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:58:01.943908 10888 raft_consensus.cc:729] T 08e01fa3f73e49a7ba32fc8e84681c0b P fc3b99856afe4cc3a86cc67240de20e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc3b99856afe4cc3a86cc67240de20e7, State: Initialized, Role: FOLLOWER
I20260811 18:58:01.944500 10888 consensus_queue.cc:260] T 08e01fa3f73e49a7ba32fc8e84681c0b P fc3b99856afe4cc3a86cc67240de20e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc3b99856afe4cc3a86cc67240de20e7" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 35525 } } peers { permanent_uuid: "aaab81ed038149258391f548b07268af" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 37583 } } peers { permanent_uuid: "fa32c49cd161420c979c48cd9fad3b05" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 41101 } }
I20260811 18:58:01.945561 10894 tablet_bootstrap.cc:654] T 08e01fa3f73e49a7ba32fc8e84681c0b P fa32c49cd161420c979c48cd9fad3b05: Neither blocks nor log segments found. Creating new log.
I20260811 18:58:01.946249 10888 ts_tablet_manager.cc:1428] T 08e01fa3f73e49a7ba32fc8e84681c0b P fc3b99856afe4cc3a86cc67240de20e7: Time spent starting tablet: real 0.011s user 0.009s sys 0.002s
I20260811 18:58:01.947479 10890 raft_consensus.cc:348] T 08e01fa3f73e49a7ba32fc8e84681c0b P aaab81ed038149258391f548b07268af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc3b99856afe4cc3a86cc67240de20e7" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 35525 } } peers { permanent_uuid: "aaab81ed038149258391f548b07268af" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 37583 } } peers { permanent_uuid: "fa32c49cd161420c979c48cd9fad3b05" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 41101 } }
I20260811 18:58:01.947918 10890 raft_consensus.cc:374] T 08e01fa3f73e49a7ba32fc8e84681c0b P aaab81ed038149258391f548b07268af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:58:01.948012 10890 raft_consensus.cc:729] T 08e01fa3f73e49a7ba32fc8e84681c0b P aaab81ed038149258391f548b07268af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aaab81ed038149258391f548b07268af, State: Initialized, Role: FOLLOWER
I20260811 18:58:01.948020 10894 log.cc:824] T 08e01fa3f73e49a7ba32fc8e84681c0b P fa32c49cd161420c979c48cd9fad3b05: Log is configured to *not* fsync() on all Append() calls
I20260811 18:58:01.948676 10890 consensus_queue.cc:260] T 08e01fa3f73e49a7ba32fc8e84681c0b P aaab81ed038149258391f548b07268af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc3b99856afe4cc3a86cc67240de20e7" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 35525 } } peers { permanent_uuid: "aaab81ed038149258391f548b07268af" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 37583 } } peers { permanent_uuid: "fa32c49cd161420c979c48cd9fad3b05" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 41101 } }
I20260811 18:58:01.952364 10592 heartbeater.cc:503] Master 127.29.112.190:37199 was elected leader, sending a full tablet report...
I20260811 18:58:01.953460 10890 ts_tablet_manager.cc:1428] T 08e01fa3f73e49a7ba32fc8e84681c0b P aaab81ed038149258391f548b07268af: Time spent starting tablet: real 0.016s user 0.014s sys 0.000s
I20260811 18:58:01.958501 10894 tablet_bootstrap.cc:492] T 08e01fa3f73e49a7ba32fc8e84681c0b P fa32c49cd161420c979c48cd9fad3b05: No bootstrap required, opened a new log
I20260811 18:58:01.958840 10894 ts_tablet_manager.cc:1397] T 08e01fa3f73e49a7ba32fc8e84681c0b P fa32c49cd161420c979c48cd9fad3b05: Time spent bootstrapping tablet: real 0.017s user 0.008s sys 0.006s
I20260811 18:58:01.966480 10894 raft_consensus.cc:348] T 08e01fa3f73e49a7ba32fc8e84681c0b P fa32c49cd161420c979c48cd9fad3b05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc3b99856afe4cc3a86cc67240de20e7" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 35525 } } peers { permanent_uuid: "aaab81ed038149258391f548b07268af" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 37583 } } peers { permanent_uuid: "fa32c49cd161420c979c48cd9fad3b05" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 41101 } }
I20260811 18:58:01.966904 10894 raft_consensus.cc:374] T 08e01fa3f73e49a7ba32fc8e84681c0b P fa32c49cd161420c979c48cd9fad3b05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:58:01.966984 10894 raft_consensus.cc:729] T 08e01fa3f73e49a7ba32fc8e84681c0b P fa32c49cd161420c979c48cd9fad3b05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa32c49cd161420c979c48cd9fad3b05, State: Initialized, Role: FOLLOWER
I20260811 18:58:01.967486 10894 consensus_queue.cc:260] T 08e01fa3f73e49a7ba32fc8e84681c0b P fa32c49cd161420c979c48cd9fad3b05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc3b99856afe4cc3a86cc67240de20e7" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 35525 } } peers { permanent_uuid: "aaab81ed038149258391f548b07268af" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 37583 } } peers { permanent_uuid: "fa32c49cd161420c979c48cd9fad3b05" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 41101 } }
I20260811 18:58:01.968652 10834 heartbeater.cc:503] Master 127.29.112.190:37199 was elected leader, sending a full tablet report...
I20260811 18:58:01.969172 10894 ts_tablet_manager.cc:1428] T 08e01fa3f73e49a7ba32fc8e84681c0b P fa32c49cd161420c979c48cd9fad3b05: Time spent starting tablet: real 0.010s user 0.008s sys 0.002s
W20260811 18:58:01.991248 10358 tablet.cc:2369] T 08e01fa3f73e49a7ba32fc8e84681c0b P fc3b99856afe4cc3a86cc67240de20e7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:58:02.025382 10901 raft_consensus.cc:482] T 08e01fa3f73e49a7ba32fc8e84681c0b P fa32c49cd161420c979c48cd9fad3b05 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:58:02.025701 10901 raft_consensus.cc:504] T 08e01fa3f73e49a7ba32fc8e84681c0b P fa32c49cd161420c979c48cd9fad3b05 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc3b99856afe4cc3a86cc67240de20e7" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 35525 } } peers { permanent_uuid: "aaab81ed038149258391f548b07268af" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 37583 } } peers { permanent_uuid: "fa32c49cd161420c979c48cd9fad3b05" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 41101 } }
I20260811 18:58:02.027367 10901 leader_election.cc:288] T 08e01fa3f73e49a7ba32fc8e84681c0b P fa32c49cd161420c979c48cd9fad3b05 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc3b99856afe4cc3a86cc67240de20e7 (127.29.112.129:35525), aaab81ed038149258391f548b07268af (127.29.112.130:37583)
I20260811 18:58:02.035166 10883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:02.027986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.131:32883 (local address 127.29.112.129:35525)
0811 18:58:02.028255 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:02.028263 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:02.028286 (+ 23us) server_negotiation.cc:407] Connection header received
0811 18:58:02.028377 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:02.028380 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:02.028477 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:02.028591 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:58:02.028921 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:02.030005 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:02.032429 (+ 2424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:02.033764 (+ 1335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:02.033924 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:02.033999 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:02.034848 (+ 849us) server_negotiation.cc:299] Negotiation successful
0811 18:58:02.034977 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113}
I20260811 18:58:02.035254 10886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:02.028363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.131:57581 (local address 127.29.112.130:37583)
0811 18:58:02.028612 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:02.028617 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:02.028635 (+ 18us) server_negotiation.cc:407] Connection header received
0811 18:58:02.028711 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:02.028718 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:02.028787 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:02.028914 (+ 127us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:58:02.029287 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:02.030271 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:02.032492 (+ 2221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:02.033783 (+ 1291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:02.033930 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:02.034005 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:02.034888 (+ 883us) server_negotiation.cc:299] Negotiation successful
0811 18:58:02.035023 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260811 18:58:02.035245 10850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:02.027750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:35525 (local address 127.29.112.131:32883)
0811 18:58:02.027931 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:02.027947 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:02.028089 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:02.028599 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:02.028602 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:02.028622 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:58:02.028780 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:02.028787 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:02.030145 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:02.030149 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:02.032214 (+ 2065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:02.032233 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:02.034023 (+ 1790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:02.034059 (+ 36us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:02.034585 (+ 526us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:02.034692 (+ 107us) client_negotiation.cc:769] Sending connection context
0811 18:58:02.034802 (+ 110us) client_negotiation.cc:240] Negotiation successful
0811 18:58:02.034944 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56,"mutex_wait_us":51}
I20260811 18:58:02.035778 10851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:02.028132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.130:37583 (local address 127.29.112.131:57581)
0811 18:58:02.028306 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:02.028322 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:02.028398 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:02.028949 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:02.028952 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:02.028970 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:58:02.029132 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:02.029142 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:02.030385 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:02.030390 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:02.032332 (+ 1942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:02.032342 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:02.033951 (+ 1609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:02.034060 (+ 109us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:02.034585 (+ 525us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:02.034693 (+ 108us) client_negotiation.cc:769] Sending connection context
0811 18:58:02.034808 (+ 115us) client_negotiation.cc:240] Negotiation successful
0811 18:58:02.035508 (+ 700us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73,"mutex_wait_us":657}
I20260811 18:58:02.036448 10265 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08e01fa3f73e49a7ba32fc8e84681c0b" candidate_uuid: "fa32c49cd161420c979c48cd9fad3b05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc3b99856afe4cc3a86cc67240de20e7" is_pre_election: true
I20260811 18:58:02.036739 10513 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08e01fa3f73e49a7ba32fc8e84681c0b" candidate_uuid: "fa32c49cd161420c979c48cd9fad3b05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aaab81ed038149258391f548b07268af" is_pre_election: true
I20260811 18:58:02.036942 10265 raft_consensus.cc:2454] T 08e01fa3f73e49a7ba32fc8e84681c0b P fc3b99856afe4cc3a86cc67240de20e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa32c49cd161420c979c48cd9fad3b05 in term 0.
I20260811 18:58:02.037220 10513 raft_consensus.cc:2454] T 08e01fa3f73e49a7ba32fc8e84681c0b P aaab81ed038149258391f548b07268af [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa32c49cd161420c979c48cd9fad3b05 in term 0.
W20260811 18:58:02.037719 10843 tablet.cc:2369] T 08e01fa3f73e49a7ba32fc8e84681c0b P fa32c49cd161420c979c48cd9fad3b05: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:58:02.037669 10650 leader_election.cc:302] T 08e01fa3f73e49a7ba32fc8e84681c0b P fa32c49cd161420c979c48cd9fad3b05 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: fa32c49cd161420c979c48cd9fad3b05, fc3b99856afe4cc3a86cc67240de20e7; no voters:
I20260811 18:58:02.038154 10901 raft_consensus.cc:2781] T 08e01fa3f73e49a7ba32fc8e84681c0b P fa32c49cd161420c979c48cd9fad3b05 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:58:02.038329 10901 raft_consensus.cc:482] T 08e01fa3f73e49a7ba32fc8e84681c0b P fa32c49cd161420c979c48cd9fad3b05 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:58:02.038458 10901 raft_consensus.cc:3037] T 08e01fa3f73e49a7ba32fc8e84681c0b P fa32c49cd161420c979c48cd9fad3b05 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:58:02.040683 10901 raft_consensus.cc:504] T 08e01fa3f73e49a7ba32fc8e84681c0b P fa32c49cd161420c979c48cd9fad3b05 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc3b99856afe4cc3a86cc67240de20e7" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 35525 } } peers { permanent_uuid: "aaab81ed038149258391f548b07268af" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 37583 } } peers { permanent_uuid: "fa32c49cd161420c979c48cd9fad3b05" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 41101 } }
I20260811 18:58:02.041597 10901 leader_election.cc:288] T 08e01fa3f73e49a7ba32fc8e84681c0b P fa32c49cd161420c979c48cd9fad3b05 [CANDIDATE]: Term 1 election: Requested vote from peers fc3b99856afe4cc3a86cc67240de20e7 (127.29.112.129:35525), aaab81ed038149258391f548b07268af (127.29.112.130:37583)
I20260811 18:58:02.042080 10265 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08e01fa3f73e49a7ba32fc8e84681c0b" candidate_uuid: "fa32c49cd161420c979c48cd9fad3b05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc3b99856afe4cc3a86cc67240de20e7"
I20260811 18:58:02.042095 10513 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08e01fa3f73e49a7ba32fc8e84681c0b" candidate_uuid: "fa32c49cd161420c979c48cd9fad3b05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aaab81ed038149258391f548b07268af"
I20260811 18:58:02.042265 10265 raft_consensus.cc:3037] T 08e01fa3f73e49a7ba32fc8e84681c0b P fc3b99856afe4cc3a86cc67240de20e7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:58:02.042297 10513 raft_consensus.cc:3037] T 08e01fa3f73e49a7ba32fc8e84681c0b P aaab81ed038149258391f548b07268af [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:58:02.044461 10265 raft_consensus.cc:2454] T 08e01fa3f73e49a7ba32fc8e84681c0b P fc3b99856afe4cc3a86cc67240de20e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa32c49cd161420c979c48cd9fad3b05 in term 1.
I20260811 18:58:02.044570 10513 raft_consensus.cc:2454] T 08e01fa3f73e49a7ba32fc8e84681c0b P aaab81ed038149258391f548b07268af [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa32c49cd161420c979c48cd9fad3b05 in term 1.
I20260811 18:58:02.044870 10650 leader_election.cc:302] T 08e01fa3f73e49a7ba32fc8e84681c0b P fa32c49cd161420c979c48cd9fad3b05 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: fa32c49cd161420c979c48cd9fad3b05, fc3b99856afe4cc3a86cc67240de20e7; no voters:
I20260811 18:58:02.045313 10901 raft_consensus.cc:2781] T 08e01fa3f73e49a7ba32fc8e84681c0b P fa32c49cd161420c979c48cd9fad3b05 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:58:02.045763 10901 raft_consensus.cc:686] T 08e01fa3f73e49a7ba32fc8e84681c0b P fa32c49cd161420c979c48cd9fad3b05 [term 1 LEADER]: Becoming Leader. State: Replica: fa32c49cd161420c979c48cd9fad3b05, State: Running, Role: LEADER
I20260811 18:58:02.046298 10901 consensus_queue.cc:237] T 08e01fa3f73e49a7ba32fc8e84681c0b P fa32c49cd161420c979c48cd9fad3b05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc3b99856afe4cc3a86cc67240de20e7" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 35525 } } peers { permanent_uuid: "aaab81ed038149258391f548b07268af" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 37583 } } peers { permanent_uuid: "fa32c49cd161420c979c48cd9fad3b05" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 41101 } }
I20260811 18:58:02.051854 8598 catalog_manager.cc:5495] T 08e01fa3f73e49a7ba32fc8e84681c0b P fa32c49cd161420c979c48cd9fad3b05 reported cstate change: term changed from 0 to 1, leader changed from <none> to fa32c49cd161420c979c48cd9fad3b05 (127.29.112.131). New cstate: current_term: 1 leader_uuid: "fa32c49cd161420c979c48cd9fad3b05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc3b99856afe4cc3a86cc67240de20e7" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 35525 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aaab81ed038149258391f548b07268af" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 37583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa32c49cd161420c979c48cd9fad3b05" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 41101 } health_report { overall_health: HEALTHY } } }
I20260811 18:58:02.102095 10887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:02.094665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53282 (local address 127.29.112.131:41101)
0811 18:58:02.094894 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:02.094901 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:02.094921 (+ 20us) server_negotiation.cc:407] Connection header received
0811 18:58:02.095074 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:02.095078 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:02.095220 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:02.095386 (+ 166us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:58:02.095743 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:02.096473 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:02.098701 (+ 2228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:02.099353 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:02.099552 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:02.100235 (+ 683us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:58:02.101104 (+ 869us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:58:02.101306 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:02.101493 (+ 187us) server_negotiation.cc:299] Negotiation successful
0811 18:58:02.101621 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 18:58:02.111694 10513 raft_consensus.cc:1261] T 08e01fa3f73e49a7ba32fc8e84681c0b P aaab81ed038149258391f548b07268af [term 1 FOLLOWER]: Refusing update from remote peer fa32c49cd161420c979c48cd9fad3b05: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:58:02.111773 10265 raft_consensus.cc:1261] T 08e01fa3f73e49a7ba32fc8e84681c0b P fc3b99856afe4cc3a86cc67240de20e7 [term 1 FOLLOWER]: Refusing update from remote peer fa32c49cd161420c979c48cd9fad3b05: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:58:02.112788 10901 consensus_queue.cc:1035] T 08e01fa3f73e49a7ba32fc8e84681c0b P fa32c49cd161420c979c48cd9fad3b05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aaab81ed038149258391f548b07268af" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 37583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 18:58:02.113082 10595 tablet.cc:2369] T 08e01fa3f73e49a7ba32fc8e84681c0b P aaab81ed038149258391f548b07268af: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:58:02.113102 10905 consensus_queue.cc:1035] T 08e01fa3f73e49a7ba32fc8e84681c0b P fa32c49cd161420c979c48cd9fad3b05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc3b99856afe4cc3a86cc67240de20e7" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 35525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:58:02.141896 10915 mvcc.cc:205] Tried to move back new op lower bound from 7317400297918066688 to 7317400297665458176. Current Snapshot: MvccSnapshot[applied={T|T < 7317400297918066688}]
I20260811 18:58:02.150149 8599 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60544:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:58:02.150457 8599 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60544:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:58:02.156857 8599 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b994bc87dbdf4d93a9b50500ad3f2aca: Sending DeleteTablet for 3 replicas of tablet 08e01fa3f73e49a7ba32fc8e84681c0b
I20260811 18:58:02.158078 10480 tablet_service.cc:1514] Processing DeleteTablet for tablet 08e01fa3f73e49a7ba32fc8e84681c0b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:58:02 UTC) from {username='oryx', principal='oryx/127.29.112.190@KRBTEST.COM'} at 127.0.0.1:60580
I20260811 18:58:02.158113 10229 tablet_service.cc:1514] Processing DeleteTablet for tablet 08e01fa3f73e49a7ba32fc8e84681c0b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:58:02 UTC) from {username='oryx', principal='oryx/127.29.112.190@KRBTEST.COM'} at 127.0.0.1:35810
I20260811 18:58:02.159080 10719 tablet_service.cc:1514] Processing DeleteTablet for tablet 08e01fa3f73e49a7ba32fc8e84681c0b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:58:02 UTC) from {username='oryx', principal='oryx/127.29.112.190@KRBTEST.COM'} at 127.0.0.1:53278
I20260811 18:58:02.172308 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 8537
I20260811 18:58:02.173301 8641 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:58:02.176819 10934 tablet_replica.cc:330] T 08e01fa3f73e49a7ba32fc8e84681c0b P fc3b99856afe4cc3a86cc67240de20e7: stopping tablet replica
I20260811 18:58:02.177387 10934 raft_consensus.cc:2229] T 08e01fa3f73e49a7ba32fc8e84681c0b P fc3b99856afe4cc3a86cc67240de20e7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:58:02.178122 10934 raft_consensus.cc:2258] T 08e01fa3f73e49a7ba32fc8e84681c0b P fc3b99856afe4cc3a86cc67240de20e7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:58:02.184304 10934 ts_tablet_manager.cc:1905] T 08e01fa3f73e49a7ba32fc8e84681c0b P fc3b99856afe4cc3a86cc67240de20e7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:58:02.187115 10935 tablet_replica.cc:330] T 08e01fa3f73e49a7ba32fc8e84681c0b P fa32c49cd161420c979c48cd9fad3b05: stopping tablet replica
I20260811 18:58:02.187547 10935 raft_consensus.cc:2229] T 08e01fa3f73e49a7ba32fc8e84681c0b P fa32c49cd161420c979c48cd9fad3b05 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:58:02.188115 10931 tablet_replica.cc:330] T 08e01fa3f73e49a7ba32fc8e84681c0b P aaab81ed038149258391f548b07268af: stopping tablet replica
I20260811 18:58:02.188328 10935 raft_consensus.cc:2258] T 08e01fa3f73e49a7ba32fc8e84681c0b P fa32c49cd161420c979c48cd9fad3b05 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:58:02.188545 10931 raft_consensus.cc:2229] T 08e01fa3f73e49a7ba32fc8e84681c0b P aaab81ed038149258391f548b07268af [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:58:02.189054 10931 raft_consensus.cc:2258] T 08e01fa3f73e49a7ba32fc8e84681c0b P aaab81ed038149258391f548b07268af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:58:02.190768 10935 ts_tablet_manager.cc:1905] T 08e01fa3f73e49a7ba32fc8e84681c0b P fa32c49cd161420c979c48cd9fad3b05: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:58:02.192039 10931 ts_tablet_manager.cc:1905] T 08e01fa3f73e49a7ba32fc8e84681c0b P aaab81ed038149258391f548b07268af: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:58:02.201242 10935 ts_tablet_manager.cc:1918] T 08e01fa3f73e49a7ba32fc8e84681c0b P fa32c49cd161420c979c48cd9fad3b05: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:58:02.201454 10935 log.cc:1196] T 08e01fa3f73e49a7ba32fc8e84681c0b P fa32c49cd161420c979c48cd9fad3b05: Deleting WAL directory at /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/ts-2/wal/wals/08e01fa3f73e49a7ba32fc8e84681c0b
I20260811 18:58:02.202134 10935 ts_tablet_manager.cc:1939] T 08e01fa3f73e49a7ba32fc8e84681c0b P fa32c49cd161420c979c48cd9fad3b05: Deleting consensus metadata
I20260811 18:58:02.208206 10934 ts_tablet_manager.cc:1918] T 08e01fa3f73e49a7ba32fc8e84681c0b P fc3b99856afe4cc3a86cc67240de20e7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:58:02.208302 10931 ts_tablet_manager.cc:1918] T 08e01fa3f73e49a7ba32fc8e84681c0b P aaab81ed038149258391f548b07268af: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:58:02.208484 10931 log.cc:1196] T 08e01fa3f73e49a7ba32fc8e84681c0b P aaab81ed038149258391f548b07268af: Deleting WAL directory at /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/ts-1/wal/wals/08e01fa3f73e49a7ba32fc8e84681c0b
I20260811 18:58:02.208595 10934 log.cc:1196] T 08e01fa3f73e49a7ba32fc8e84681c0b P fc3b99856afe4cc3a86cc67240de20e7: Deleting WAL directory at /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/ts-0/wal/wals/08e01fa3f73e49a7ba32fc8e84681c0b
I20260811 18:58:02.209048 10931 ts_tablet_manager.cc:1939] T 08e01fa3f73e49a7ba32fc8e84681c0b P aaab81ed038149258391f548b07268af: Deleting consensus metadata
I20260811 18:58:02.209273 10934 ts_tablet_manager.cc:1939] T 08e01fa3f73e49a7ba32fc8e84681c0b P fc3b99856afe4cc3a86cc67240de20e7: Deleting consensus metadata
I20260811 18:58:02.214390 10859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:02.161177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56298 (local address 127.29.112.189:45725)
0811 18:58:02.161386 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:02.161394 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:02.171607 (+ 10213us) server_negotiation.cc:407] Connection header received
0811 18:58:02.172397 (+ 790us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:02.172405 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:02.172514 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:02.172604 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:02.178515 (+ 5911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:02.180170 (+ 1655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:02.189419 (+ 9249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:02.190136 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:02.190289 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:02.207224 (+ 16935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:58:02.207231 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:58:02.207237 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:58:02.207288 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:58:02.211200 (+ 3912us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:02.212116 (+ 916us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:02.212123 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:02.212127 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:02.212286 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:02.212741 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:02.212746 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:02.212750 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:02.213121 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:58:02.213371 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:02.213792 (+ 421us) server_negotiation.cc:299] Negotiation successful
0811 18:58:02.214022 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 18:58:02.231920 10861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:02.171024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47644 (local address 127.29.112.188:44651)
0811 18:58:02.171230 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:02.171236 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:02.214500 (+ 43264us) server_negotiation.cc:407] Connection header received
0811 18:58:02.214871 (+ 371us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:02.214878 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:02.215005 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:02.215079 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:02.216990 (+ 1911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:02.218126 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:02.220303 (+ 2177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:02.221205 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:02.221358 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:02.224971 (+ 3613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:58:02.224979 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:58:02.224985 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:58:02.225042 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:58:02.228900 (+ 3858us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:02.229723 (+ 823us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:02.229729 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:02.229734 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:02.229900 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:02.230335 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:02.230342 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:02.230346 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:02.230760 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:58:02.230989 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:02.231374 (+ 385us) server_negotiation.cc:299] Negotiation successful
0811 18:58:02.231590 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 18:58:02.339277 8571 catalog_manager.cc:4841] TS fc3b99856afe4cc3a86cc67240de20e7 (127.29.112.129:35525): tablet 08e01fa3f73e49a7ba32fc8e84681c0b (table test-table [id=cc0bb03df40c44fbab8454dc4b84ad64]) successfully deleted
I20260811 18:58:02.339452 8567 catalog_manager.cc:4841] TS fa32c49cd161420c979c48cd9fad3b05 (127.29.112.131:41101): tablet 08e01fa3f73e49a7ba32fc8e84681c0b (table test-table [id=cc0bb03df40c44fbab8454dc4b84ad64]) successfully deleted
I20260811 18:58:02.339680 8571 catalog_manager.cc:4841] TS aaab81ed038149258391f548b07268af (127.29.112.130:37583): tablet 08e01fa3f73e49a7ba32fc8e84681c0b (table test-table [id=cc0bb03df40c44fbab8454dc4b84ad64]) successfully deleted
I20260811 18:58:02.340049 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 8537
I20260811 18:58:02.412494 10861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:02.383896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47648 (local address 127.29.112.188:44651)
0811 18:58:02.384131 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:02.384139 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:02.396541 (+ 12402us) server_negotiation.cc:407] Connection header received
0811 18:58:02.396723 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:02.396728 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:02.396803 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:02.396913 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:02.398824 (+ 1911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:02.399949 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:02.401723 (+ 1774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:02.402134 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:02.402271 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:02.406426 (+ 4155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:58:02.406433 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:58:02.406438 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:58:02.406480 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:58:02.409544 (+ 3064us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:02.410206 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:02.410211 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:02.410219 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:02.410345 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:02.410803 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:02.410807 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:02.410809 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:02.411071 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:58:02.411311 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:02.411728 (+ 417us) server_negotiation.cc:299] Negotiation successful
0811 18:58:02.411944 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 18:58:02.415256 10859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:02.383838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56300 (local address 127.29.112.189:45725)
0811 18:58:02.384029 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:02.384037 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:02.395705 (+ 11668us) server_negotiation.cc:407] Connection header received
0811 18:58:02.396060 (+ 355us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:02.396065 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:02.396170 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:02.396291 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:02.398406 (+ 2115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:02.399642 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:02.401557 (+ 1915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:02.402455 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:02.402733 (+ 278us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:02.407243 (+ 4510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:58:02.407250 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:58:02.407255 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:58:02.407296 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:58:02.411078 (+ 3782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:02.412738 (+ 1660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:02.412743 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:02.412746 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:02.412888 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:02.413837 (+ 949us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:02.413843 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:02.413846 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:02.414116 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:58:02.414367 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:02.414732 (+ 365us) server_negotiation.cc:299] Negotiation successful
0811 18:58:02.414958 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 18:58:03.146950 10953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:03.146054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37199 (local address 127.29.112.129:53423)
0811 18:58:03.146637 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:03.146757 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:37199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":266,"threads_started":1}
W20260811 18:58:03.147568 10347 heartbeater.cc:644] Failed to heartbeat to 127.29.112.190:37199 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.29.112.190:37199: connect: Connection refused (error 111)
I20260811 18:58:03.148782 10954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:03.147701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37199 (local address 127.29.112.131:34181)
0811 18:58:03.148255 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:03.148430 (+ 175us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:37199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":272,"threads_started":1}
W20260811 18:58:03.149223 10834 heartbeater.cc:644] Failed to heartbeat to 127.29.112.190:37199 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.29.112.190:37199: connect: Connection refused (error 111)
I20260811 18:58:03.164609 10955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:03.163828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37199 (local address 127.29.112.130:50779)
0811 18:58:03.164323 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:03.164441 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:37199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":280,"threads_started":1}
W20260811 18:58:03.165024 10592 heartbeater.cc:644] Failed to heartbeat to 127.29.112.190:37199 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.29.112.190:37199: connect: Connection refused (error 111)
I20260811 18:58:03.657692 10959 raft_consensus.cc:482] T 00000000000000000000000000000000 P ea739164f1b344099a538c6c91d0ada6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b994bc87dbdf4d93a9b50500ad3f2aca)
I20260811 18:58:03.658165 10959 raft_consensus.cc:504] T 00000000000000000000000000000000 P ea739164f1b344099a538c6c91d0ada6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "b994bc87dbdf4d93a9b50500ad3f2aca" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 37199 } } peers { permanent_uuid: "ea739164f1b344099a538c6c91d0ada6" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 45725 } attrs { promote: false } } peers { permanent_uuid: "4198b17e42c94aac90009c1bc0970564" member_type: VOTER last_known_addr { host: "127.29.112.188" port: 44651 } attrs { promote: false } }
I20260811 18:58:03.658852 10960 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4198b17e42c94aac90009c1bc0970564 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b994bc87dbdf4d93a9b50500ad3f2aca)
I20260811 18:58:03.659161 10960 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4198b17e42c94aac90009c1bc0970564 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "b994bc87dbdf4d93a9b50500ad3f2aca" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 37199 } } peers { permanent_uuid: "ea739164f1b344099a538c6c91d0ada6" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 45725 } attrs { promote: false } } peers { permanent_uuid: "4198b17e42c94aac90009c1bc0970564" member_type: VOTER last_known_addr { host: "127.29.112.188" port: 44651 } attrs { promote: false } }
I20260811 18:58:03.660158 10959 leader_election.cc:288] T 00000000000000000000000000000000 P ea739164f1b344099a538c6c91d0ada6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b994bc87dbdf4d93a9b50500ad3f2aca (127.29.112.190:37199), 4198b17e42c94aac90009c1bc0970564 (127.29.112.188:44651)
I20260811 18:58:03.660755 10960 leader_election.cc:288] T 00000000000000000000000000000000 P 4198b17e42c94aac90009c1bc0970564 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b994bc87dbdf4d93a9b50500ad3f2aca (127.29.112.190:37199), ea739164f1b344099a538c6c91d0ada6 (127.29.112.189:45725)
I20260811 18:58:03.662057 10966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:03.660804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37199 (local address 127.0.0.1:60916)
0811 18:58:03.661448 (+ 644us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:03.661698 (+ 250us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:37199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":360,"threads_started":1}
W20260811 18:58:03.662776 10022 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.112.190:37199: connect: Connection refused (error 111)
I20260811 18:58:03.663661 10964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:03.660416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37199 (local address 127.0.0.1:60904)
0811 18:58:03.663106 (+ 2690us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:03.663390 (+ 284us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:37199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2540,"thread_start_us":2998,"threads_started":1}
W20260811 18:58:03.664036 9665 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.112.190:37199: connect: Connection refused (error 111)
I20260811 18:58:03.674649 10966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:03.674151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37199 (local address 127.0.0.1:60928)
0811 18:58:03.674294 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:03.674401 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:37199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260811 18:58:03.674903 10965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:03.674188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37199 (local address 127.0.0.1:60944)
0811 18:58:03.674421 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:03.674632 (+ 211us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:37199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
W20260811 18:58:03.675159 9665 leader_election.cc:334] T 00000000000000000000000000000000 P ea739164f1b344099a538c6c91d0ada6 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer b994bc87dbdf4d93a9b50500ad3f2aca (127.29.112.190:37199): Network error: Client connection negotiation failed: client connection to 127.29.112.190:37199: connect: Connection refused (error 111)
W20260811 18:58:03.675558 10022 leader_election.cc:334] T 00000000000000000000000000000000 P 4198b17e42c94aac90009c1bc0970564 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer b994bc87dbdf4d93a9b50500ad3f2aca (127.29.112.190:37199): Network error: Client connection negotiation failed: client connection to 127.29.112.190:37199: connect: Connection refused (error 111)
I20260811 18:58:03.680956 10964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:03.660438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.188:44651 (local address 127.0.0.1:47654)
0811 18:58:03.663961 (+ 3523us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:03.663985 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:03.664165 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:03.674705 (+ 10540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:03.674712 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:03.674747 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:58:03.675120 (+ 373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:03.675139 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:03.677403 (+ 2264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:03.677414 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:03.678829 (+ 1415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:03.678842 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:03.680018 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:03.680024 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:03.680265 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:03.680299 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 18:58:03.680428 (+ 129us) client_negotiation.cc:240] Negotiation successful
0811 18:58:03.680582 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3427,"thread_start_us":13220,"threads_started":1}
I20260811 18:58:03.681109 10966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:03.674663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.189:45725 (local address 127.0.0.1:56316)
0811 18:58:03.674837 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:03.674859 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:03.674984 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:03.676159 (+ 1175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:03.676164 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:03.676198 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:58:03.676474 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:03.676484 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:03.677612 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:03.677620 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:03.678929 (+ 1309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:03.678941 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:03.680146 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:03.680152 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:03.680310 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:03.680332 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 18:58:03.680469 (+ 137us) client_negotiation.cc:240] Negotiation successful
0811 18:58:03.680732 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":81,"mutex_wait_us":58,"thread_start_us":926,"threads_started":1}
I20260811 18:58:03.681402 10973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:03.674340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56316 (local address 127.29.112.189:45725)
0811 18:58:03.675873 (+ 1533us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:03.675878 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:03.675903 (+ 25us) server_negotiation.cc:407] Connection header received
0811 18:58:03.675962 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:03.675965 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:03.676015 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:03.676136 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:58:03.676636 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:03.677453 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:03.679082 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:03.679870 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:03.680147 (+ 277us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:03.680217 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:03.680608 (+ 391us) server_negotiation.cc:299] Negotiation successful
0811 18:58:03.680776 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":1414,"thread_start_us":1449,"threads_started":1}
I20260811 18:58:03.681661 10968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:03.660839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47654 (local address 127.29.112.188:44651)
0811 18:58:03.674265 (+ 13426us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:03.674272 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:03.674293 (+ 21us) server_negotiation.cc:407] Connection header received
0811 18:58:03.674374 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:03.674380 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:03.674467 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:03.675276 (+ 809us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:58:03.676338 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:03.677230 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:03.678983 (+ 1753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:03.679804 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:03.680023 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:03.680113 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:03.680454 (+ 341us) server_negotiation.cc:299] Negotiation successful
0811 18:58:03.681309 (+ 855us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":64,"server-negotiator.queue_time_us":13275,"thread_start_us":13149,"threads_started":1}
I20260811 18:58:03.682250 9704 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4198b17e42c94aac90009c1bc0970564" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "ea739164f1b344099a538c6c91d0ada6" is_pre_election: true
I20260811 18:58:03.682272 10052 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ea739164f1b344099a538c6c91d0ada6" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "4198b17e42c94aac90009c1bc0970564" is_pre_election: true
I20260811 18:58:03.682664 10052 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4198b17e42c94aac90009c1bc0970564 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ea739164f1b344099a538c6c91d0ada6 in term 1.
I20260811 18:58:03.682662 9704 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ea739164f1b344099a538c6c91d0ada6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4198b17e42c94aac90009c1bc0970564 in term 1.
I20260811 18:58:03.683275 9661 leader_election.cc:302] T 00000000000000000000000000000000 P ea739164f1b344099a538c6c91d0ada6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4198b17e42c94aac90009c1bc0970564, ea739164f1b344099a538c6c91d0ada6; no voters: b994bc87dbdf4d93a9b50500ad3f2aca
I20260811 18:58:03.683269 10018 leader_election.cc:302] T 00000000000000000000000000000000 P 4198b17e42c94aac90009c1bc0970564 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4198b17e42c94aac90009c1bc0970564, ea739164f1b344099a538c6c91d0ada6; no voters: b994bc87dbdf4d93a9b50500ad3f2aca
I20260811 18:58:03.683682 10959 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ea739164f1b344099a538c6c91d0ada6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260811 18:58:03.683705 10960 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4198b17e42c94aac90009c1bc0970564 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260811 18:58:03.683825 10959 raft_consensus.cc:482] T 00000000000000000000000000000000 P ea739164f1b344099a538c6c91d0ada6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader b994bc87dbdf4d93a9b50500ad3f2aca)
I20260811 18:58:03.683859 10960 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4198b17e42c94aac90009c1bc0970564 [term 1 FOLLOWER]: Starting leader election (detected failure of leader b994bc87dbdf4d93a9b50500ad3f2aca)
I20260811 18:58:03.683919 10959 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ea739164f1b344099a538c6c91d0ada6 [term 1 FOLLOWER]: Advancing to term 2
I20260811 18:58:03.683974 10960 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4198b17e42c94aac90009c1bc0970564 [term 1 FOLLOWER]: Advancing to term 2
I20260811 18:58:03.686005 10959 raft_consensus.cc:504] T 00000000000000000000000000000000 P ea739164f1b344099a538c6c91d0ada6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "b994bc87dbdf4d93a9b50500ad3f2aca" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 37199 } } peers { permanent_uuid: "ea739164f1b344099a538c6c91d0ada6" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 45725 } attrs { promote: false } } peers { permanent_uuid: "4198b17e42c94aac90009c1bc0970564" member_type: VOTER last_known_addr { host: "127.29.112.188" port: 44651 } attrs { promote: false } }
I20260811 18:58:03.686142 10960 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4198b17e42c94aac90009c1bc0970564 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "b994bc87dbdf4d93a9b50500ad3f2aca" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 37199 } } peers { permanent_uuid: "ea739164f1b344099a538c6c91d0ada6" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 45725 } attrs { promote: false } } peers { permanent_uuid: "4198b17e42c94aac90009c1bc0970564" member_type: VOTER last_known_addr { host: "127.29.112.188" port: 44651 } attrs { promote: false } }
I20260811 18:58:03.686779 10959 leader_election.cc:288] T 00000000000000000000000000000000 P ea739164f1b344099a538c6c91d0ada6 [CANDIDATE]: Term 2 election: Requested vote from peers b994bc87dbdf4d93a9b50500ad3f2aca (127.29.112.190:37199), 4198b17e42c94aac90009c1bc0970564 (127.29.112.188:44651)
I20260811 18:58:03.686929 10960 leader_election.cc:288] T 00000000000000000000000000000000 P 4198b17e42c94aac90009c1bc0970564 [CANDIDATE]: Term 2 election: Requested vote from peers b994bc87dbdf4d93a9b50500ad3f2aca (127.29.112.190:37199), ea739164f1b344099a538c6c91d0ada6 (127.29.112.189:45725)
I20260811 18:58:03.687386 10964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:03.686829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37199 (local address 127.0.0.1:60958)
0811 18:58:03.687209 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:03.687275 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:37199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":282}
I20260811 18:58:03.687589 10966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:03.687162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37199 (local address 127.0.0.1:60968)
0811 18:58:03.687327 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:03.687433 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:37199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260811 18:58:03.688271 9704 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4198b17e42c94aac90009c1bc0970564" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "ea739164f1b344099a538c6c91d0ada6"
I20260811 18:58:03.688273 10052 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ea739164f1b344099a538c6c91d0ada6" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "4198b17e42c94aac90009c1bc0970564"
I20260811 18:58:03.688566 9704 raft_consensus.cc:2379] T 00000000000000000000000000000000 P ea739164f1b344099a538c6c91d0ada6 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 4198b17e42c94aac90009c1bc0970564 in current term 2: Already voted for candidate ea739164f1b344099a538c6c91d0ada6 in this term.
I20260811 18:58:03.688575 10052 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 4198b17e42c94aac90009c1bc0970564 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate ea739164f1b344099a538c6c91d0ada6 in current term 2: Already voted for candidate 4198b17e42c94aac90009c1bc0970564 in this term.
I20260811 18:58:03.688864 10964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:03.688407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37199 (local address 127.0.0.1:60984)
0811 18:58:03.688621 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:03.688728 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:37199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
W20260811 18:58:03.689087 9665 leader_election.cc:334] T 00000000000000000000000000000000 P ea739164f1b344099a538c6c91d0ada6 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer b994bc87dbdf4d93a9b50500ad3f2aca (127.29.112.190:37199): Network error: Client connection negotiation failed: client connection to 127.29.112.190:37199: connect: Connection refused (error 111)
I20260811 18:58:03.689163 10966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:03.688334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37199 (local address 127.0.0.1:60980)
0811 18:58:03.688913 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:03.688994 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:37199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":449}
I20260811 18:58:03.689275 9661 leader_election.cc:302] T 00000000000000000000000000000000 P ea739164f1b344099a538c6c91d0ada6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ea739164f1b344099a538c6c91d0ada6; no voters: 4198b17e42c94aac90009c1bc0970564, b994bc87dbdf4d93a9b50500ad3f2aca
W20260811 18:58:03.689347 10022 leader_election.cc:334] T 00000000000000000000000000000000 P 4198b17e42c94aac90009c1bc0970564 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer b994bc87dbdf4d93a9b50500ad3f2aca (127.29.112.190:37199): Network error: Client connection negotiation failed: client connection to 127.29.112.190:37199: connect: Connection refused (error 111)
I20260811 18:58:03.689543 10959 raft_consensus.cc:2726] T 00000000000000000000000000000000 P ea739164f1b344099a538c6c91d0ada6 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260811 18:58:03.689581 10018 leader_election.cc:302] T 00000000000000000000000000000000 P 4198b17e42c94aac90009c1bc0970564 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4198b17e42c94aac90009c1bc0970564; no voters: b994bc87dbdf4d93a9b50500ad3f2aca, ea739164f1b344099a538c6c91d0ada6
I20260811 18:58:03.689857 10960 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 4198b17e42c94aac90009c1bc0970564 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260811 18:58:04.149209 10976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:04.148301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37199 (local address 127.29.112.129:41977)
0811 18:58:04.148889 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:04.149005 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:37199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":294,"threads_started":1}
I20260811 18:58:04.150674 10977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:04.149826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37199 (local address 127.29.112.131:36351)
0811 18:58:04.150369 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:04.150446 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:37199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":344,"threads_started":1}
I20260811 18:58:04.166368 10978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:04.165646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37199 (local address 127.29.112.130:60619)
0811 18:58:04.166141 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:04.166218 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:37199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":296,"threads_started":1}
I20260811 18:58:05.150870 10983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:05.150105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37199 (local address 127.29.112.129:47169)
0811 18:58:05.150579 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:05.150694 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:37199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":221,"threads_started":1}
I20260811 18:58:05.152004 10986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:05.151383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37199 (local address 127.29.112.131:58607)
0811 18:58:05.151772 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:05.151849 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:37199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":211,"threads_started":1}
I20260811 18:58:05.167889 10989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:05.167219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37199 (local address 127.29.112.130:55079)
0811 18:58:05.167656 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:05.167733 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:37199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":254,"threads_started":1}
I20260811 18:58:05.192867 10990 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4198b17e42c94aac90009c1bc0970564 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:58:05.193137 10990 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4198b17e42c94aac90009c1bc0970564 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "b994bc87dbdf4d93a9b50500ad3f2aca" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 37199 } } peers { permanent_uuid: "ea739164f1b344099a538c6c91d0ada6" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 45725 } attrs { promote: false } } peers { permanent_uuid: "4198b17e42c94aac90009c1bc0970564" member_type: VOTER last_known_addr { host: "127.29.112.188" port: 44651 } attrs { promote: false } }
I20260811 18:58:05.193931 10990 leader_election.cc:288] T 00000000000000000000000000000000 P 4198b17e42c94aac90009c1bc0970564 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b994bc87dbdf4d93a9b50500ad3f2aca (127.29.112.190:37199), ea739164f1b344099a538c6c91d0ada6 (127.29.112.189:45725)
I20260811 18:58:05.194656 9704 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4198b17e42c94aac90009c1bc0970564" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "ea739164f1b344099a538c6c91d0ada6" is_pre_election: true
I20260811 18:58:05.195297 9704 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ea739164f1b344099a538c6c91d0ada6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4198b17e42c94aac90009c1bc0970564 in term 2.
I20260811 18:58:05.195415 10992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:05.194084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37199 (local address 127.0.0.1:32970)
0811 18:58:05.194700 (+ 616us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:05.194993 (+ 293us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:37199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":303,"threads_started":1}
I20260811 18:58:05.196081 10018 leader_election.cc:302] T 00000000000000000000000000000000 P 4198b17e42c94aac90009c1bc0970564 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4198b17e42c94aac90009c1bc0970564, ea739164f1b344099a538c6c91d0ada6; no voters:
I20260811 18:58:05.196393 10990 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4198b17e42c94aac90009c1bc0970564 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260811 18:58:05.196554 10990 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4198b17e42c94aac90009c1bc0970564 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:58:05.196681 10990 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4198b17e42c94aac90009c1bc0970564 [term 2 FOLLOWER]: Advancing to term 3
I20260811 18:58:05.197099 10992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:05.196582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37199 (local address 127.0.0.1:32980)
0811 18:58:05.196792 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:05.196885 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:37199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
W20260811 18:58:05.197300 10022 leader_election.cc:334] T 00000000000000000000000000000000 P 4198b17e42c94aac90009c1bc0970564 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer b994bc87dbdf4d93a9b50500ad3f2aca (127.29.112.190:37199): Network error: Client connection negotiation failed: client connection to 127.29.112.190:37199: connect: Connection refused (error 111)
I20260811 18:58:05.199234 10990 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4198b17e42c94aac90009c1bc0970564 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "b994bc87dbdf4d93a9b50500ad3f2aca" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 37199 } } peers { permanent_uuid: "ea739164f1b344099a538c6c91d0ada6" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 45725 } attrs { promote: false } } peers { permanent_uuid: "4198b17e42c94aac90009c1bc0970564" member_type: VOTER last_known_addr { host: "127.29.112.188" port: 44651 } attrs { promote: false } }
I20260811 18:58:05.199853 10990 leader_election.cc:288] T 00000000000000000000000000000000 P 4198b17e42c94aac90009c1bc0970564 [CANDIDATE]: Term 3 election: Requested vote from peers b994bc87dbdf4d93a9b50500ad3f2aca (127.29.112.190:37199), ea739164f1b344099a538c6c91d0ada6 (127.29.112.189:45725)
I20260811 18:58:05.200368 10992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:05.200032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37199 (local address 127.0.0.1:32992)
0811 18:58:05.200179 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:05.200243 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:37199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260811 18:58:05.200343 9704 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4198b17e42c94aac90009c1bc0970564" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "ea739164f1b344099a538c6c91d0ada6"
I20260811 18:58:05.200762 9704 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ea739164f1b344099a538c6c91d0ada6 [term 2 FOLLOWER]: Advancing to term 3
I20260811 18:58:05.201277 10992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:05.200992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37199 (local address 127.0.0.1:32996)
0811 18:58:05.201111 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:05.201181 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:37199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260811 18:58:05.201445 10022 leader_election.cc:334] T 00000000000000000000000000000000 P 4198b17e42c94aac90009c1bc0970564 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer b994bc87dbdf4d93a9b50500ad3f2aca (127.29.112.190:37199): Network error: Client connection negotiation failed: client connection to 127.29.112.190:37199: connect: Connection refused (error 111)
I20260811 18:58:05.203289 9704 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ea739164f1b344099a538c6c91d0ada6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4198b17e42c94aac90009c1bc0970564 in term 3.
I20260811 18:58:05.203745 10018 leader_election.cc:302] T 00000000000000000000000000000000 P 4198b17e42c94aac90009c1bc0970564 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4198b17e42c94aac90009c1bc0970564, ea739164f1b344099a538c6c91d0ada6; no voters: b994bc87dbdf4d93a9b50500ad3f2aca
I20260811 18:58:05.204041 10990 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4198b17e42c94aac90009c1bc0970564 [term 3 FOLLOWER]: Leader election won for term 3
I20260811 18:58:05.204494 10990 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4198b17e42c94aac90009c1bc0970564 [term 3 LEADER]: Becoming Leader. State: Replica: 4198b17e42c94aac90009c1bc0970564, State: Running, Role: LEADER
I20260811 18:58:05.204906 10990 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4198b17e42c94aac90009c1bc0970564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "b994bc87dbdf4d93a9b50500ad3f2aca" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 37199 } } peers { permanent_uuid: "ea739164f1b344099a538c6c91d0ada6" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 45725 } attrs { promote: false } } peers { permanent_uuid: "4198b17e42c94aac90009c1bc0970564" member_type: VOTER last_known_addr { host: "127.29.112.188" port: 44651 } attrs { promote: false } }
I20260811 18:58:05.206636 10995 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4198b17e42c94aac90009c1bc0970564 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4198b17e42c94aac90009c1bc0970564. Latest consensus state: current_term: 3 leader_uuid: "4198b17e42c94aac90009c1bc0970564" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "b994bc87dbdf4d93a9b50500ad3f2aca" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 37199 } } peers { permanent_uuid: "ea739164f1b344099a538c6c91d0ada6" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 45725 } attrs { promote: false } } peers { permanent_uuid: "4198b17e42c94aac90009c1bc0970564" member_type: VOTER last_known_addr { host: "127.29.112.188" port: 44651 } attrs { promote: false } } }
I20260811 18:58:05.207051 10995 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4198b17e42c94aac90009c1bc0970564 [sys.catalog]: This master's current role is: LEADER
I20260811 18:58:05.207719 10998 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:58:05.211936 10998 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:58:05.212867 10998 catalog_manager.cc:1259] Loaded cluster ID: ddf73710ba7c49fbb2a6a28cae586e9c
I20260811 18:58:05.212971 10998 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:58:05.214577 10998 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:58:05.215533 10998 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 4198b17e42c94aac90009c1bc0970564: Loaded TSK: 0
I20260811 18:58:05.216455 10998 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:58:05.296015 10039 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47648:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:58:05.303401 10039 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:58:05.308589 10992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:05.307991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37199 (local address 127.0.0.1:33026)
0811 18:58:05.308286 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:05.308404 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:37199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163}
I20260811 18:58:05.308552 9704 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ea739164f1b344099a538c6c91d0ada6 [term 3 FOLLOWER]: Refusing update from remote peer 4198b17e42c94aac90009c1bc0970564: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260811 18:58:05.309338 10995 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4198b17e42c94aac90009c1bc0970564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea739164f1b344099a538c6c91d0ada6" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 45725 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260811 18:58:05.309646 10992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:05.309247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37199 (local address 127.0.0.1:33038)
0811 18:58:05.309440 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:05.309505 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:37199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260811 18:58:05.309821 10022 consensus_peers.cc:487] T 00000000000000000000000000000000 P 4198b17e42c94aac90009c1bc0970564 -> Peer b994bc87dbdf4d93a9b50500ad3f2aca (127.29.112.190:37199): Couldn't send request to peer b994bc87dbdf4d93a9b50500ad3f2aca. Status: Network error: Client connection negotiation failed: client connection to 127.29.112.190:37199: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 18:58:05.312712 11000 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea739164f1b344099a538c6c91d0ada6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4198b17e42c94aac90009c1bc0970564. Latest consensus state: current_term: 3 leader_uuid: "4198b17e42c94aac90009c1bc0970564" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "b994bc87dbdf4d93a9b50500ad3f2aca" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 37199 } } peers { permanent_uuid: "ea739164f1b344099a538c6c91d0ada6" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 45725 } attrs { promote: false } } peers { permanent_uuid: "4198b17e42c94aac90009c1bc0970564" member_type: VOTER last_known_addr { host: "127.29.112.188" port: 44651 } attrs { promote: false } } }
I20260811 18:58:05.313102 11000 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea739164f1b344099a538c6c91d0ada6 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:58:05.314473 10995 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4198b17e42c94aac90009c1bc0970564 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "4198b17e42c94aac90009c1bc0970564" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "b994bc87dbdf4d93a9b50500ad3f2aca" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 37199 } } peers { permanent_uuid: "ea739164f1b344099a538c6c91d0ada6" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 45725 } attrs { promote: false } } peers { permanent_uuid: "4198b17e42c94aac90009c1bc0970564" member_type: VOTER last_known_addr { host: "127.29.112.188" port: 44651 } attrs { promote: false } } }
I20260811 18:58:05.314797 10995 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4198b17e42c94aac90009c1bc0970564 [sys.catalog]: This master's current role is: LEADER
I20260811 18:58:05.315230 11000 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea739164f1b344099a538c6c91d0ada6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "4198b17e42c94aac90009c1bc0970564" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "b994bc87dbdf4d93a9b50500ad3f2aca" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 37199 } } peers { permanent_uuid: "ea739164f1b344099a538c6c91d0ada6" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 45725 } attrs { promote: false } } peers { permanent_uuid: "4198b17e42c94aac90009c1bc0970564" member_type: VOTER last_known_addr { host: "127.29.112.188" port: 44651 } attrs { promote: false } } }
I20260811 18:58:05.315466 11000 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea739164f1b344099a538c6c91d0ada6 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:58:05.332741 10992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:05.323752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:35525 (local address 127.0.0.1:35812)
0811 18:58:05.323889 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:05.323907 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:05.324016 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:05.324819 (+ 803us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:05.324826 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:05.324858 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:58:05.325220 (+ 362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:05.325233 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:05.327109 (+ 1876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:05.327118 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:05.329576 (+ 2458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:05.329596 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:05.331725 (+ 2129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:05.331734 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:05.331975 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:05.332026 (+ 51us) client_negotiation.cc:769] Sending connection context
0811 18:58:05.332103 (+ 77us) client_negotiation.cc:240] Negotiation successful
0811 18:58:05.332284 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260811 18:58:05.334227 11013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:05.323939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35812 (local address 127.29.112.129:35525)
0811 18:58:05.324451 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:05.324458 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:05.324476 (+ 18us) server_negotiation.cc:407] Connection header received
0811 18:58:05.324561 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:05.324567 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:05.324658 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:05.325366 (+ 708us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:58:05.325582 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:05.326933 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:05.329824 (+ 2891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:05.331466 (+ 1642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:05.333416 (+ 1950us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:05.333541 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:05.333601 (+ 60us) server_negotiation.cc:299] Negotiation successful
0811 18:58:05.333775 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":259,"threads_started":1}
I20260811 18:58:05.341234 10229 tablet_service.cc:1467] Processing CreateTablet for tablet ea8cc115ceb94171b23de881aa4d469e (DEFAULT_TABLE table=test-table [id=e076620853a34ea5b99f3fc0802cde8c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:58:05.341786 10229 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea8cc115ceb94171b23de881aa4d469e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:58:05.346679 11017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:05.325020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.131:41101 (local address 127.0.0.1:53292)
0811 18:58:05.339985 (+ 14965us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:05.340009 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:05.340121 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:05.340524 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:05.340529 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:05.340553 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:58:05.340904 (+ 351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:05.340914 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:05.342287 (+ 1373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:05.342293 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:05.343955 (+ 1662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:05.343967 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:05.345883 (+ 1916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:05.345889 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:05.346067 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:05.346099 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 18:58:05.346185 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 18:58:05.346311 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14837,"thread_start_us":14689,"threads_started":1}
I20260811 18:58:05.347394 10992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:05.324612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.130:37583 (local address 127.0.0.1:60586)
0811 18:58:05.333002 (+ 8390us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:05.333024 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:05.333136 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:05.341497 (+ 8361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:05.341502 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:05.341529 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:58:05.341768 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:05.341784 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:05.343194 (+ 1410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:05.343201 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:05.345220 (+ 2019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:05.345235 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:05.346868 (+ 1633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:05.346875 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:05.347044 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:05.347066 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 18:58:05.347118 (+ 52us) client_negotiation.cc:240] Negotiation successful
0811 18:58:05.347196 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8266,"thread_start_us":15703,"threads_started":1}
I20260811 18:58:05.347460 11020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:05.325422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53292 (local address 127.29.112.131:41101)
0811 18:58:05.340270 (+ 14848us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:05.340274 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:05.340289 (+ 15us) server_negotiation.cc:407] Connection header received
0811 18:58:05.340344 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:05.340347 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:05.340401 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:05.340519 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:58:05.341132 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:05.342127 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:05.344184 (+ 2057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:05.345535 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:05.345691 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:05.345767 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:05.346672 (+ 905us) server_negotiation.cc:299] Negotiation successful
0811 18:58:05.347051 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13425,"thread_start_us":13298,"threads_started":1}
I20260811 18:58:05.348306 10719 tablet_service.cc:1467] Processing CreateTablet for tablet ea8cc115ceb94171b23de881aa4d469e (DEFAULT_TABLE table=test-table [id=e076620853a34ea5b99f3fc0802cde8c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:58:05.348587 11016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:05.324763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60586 (local address 127.29.112.130:37583)
0811 18:58:05.341124 (+ 16361us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:05.341130 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:05.341147 (+ 17us) server_negotiation.cc:407] Connection header received
0811 18:58:05.341222 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:05.341229 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:05.341309 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:05.341438 (+ 129us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:58:05.341937 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:05.343040 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:05.345384 (+ 2344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:05.346677 (+ 1293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:05.347774 (+ 1097us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:05.347877 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:05.347936 (+ 59us) server_negotiation.cc:299] Negotiation successful
0811 18:58:05.348092 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16224,"thread_start_us":16087,"threads_started":1}
I20260811 18:58:05.348766 10719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea8cc115ceb94171b23de881aa4d469e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:58:05.349551 10480 tablet_service.cc:1467] Processing CreateTablet for tablet ea8cc115ceb94171b23de881aa4d469e (DEFAULT_TABLE table=test-table [id=e076620853a34ea5b99f3fc0802cde8c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:58:05.349754 11024 tablet_bootstrap.cc:492] T ea8cc115ceb94171b23de881aa4d469e P fc3b99856afe4cc3a86cc67240de20e7: Bootstrap starting.
I20260811 18:58:05.349968 10480 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea8cc115ceb94171b23de881aa4d469e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:58:05.351820 11024 tablet_bootstrap.cc:654] T ea8cc115ceb94171b23de881aa4d469e P fc3b99856afe4cc3a86cc67240de20e7: Neither blocks nor log segments found. Creating new log.
I20260811 18:58:05.354040 11024 tablet_bootstrap.cc:492] T ea8cc115ceb94171b23de881aa4d469e P fc3b99856afe4cc3a86cc67240de20e7: No bootstrap required, opened a new log
I20260811 18:58:05.354226 11024 ts_tablet_manager.cc:1397] T ea8cc115ceb94171b23de881aa4d469e P fc3b99856afe4cc3a86cc67240de20e7: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260811 18:58:05.355048 11024 raft_consensus.cc:348] T ea8cc115ceb94171b23de881aa4d469e P fc3b99856afe4cc3a86cc67240de20e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc3b99856afe4cc3a86cc67240de20e7" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 35525 } } peers { permanent_uuid: "aaab81ed038149258391f548b07268af" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 37583 } } peers { permanent_uuid: "fa32c49cd161420c979c48cd9fad3b05" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 41101 } }
I20260811 18:58:05.355374 11024 raft_consensus.cc:374] T ea8cc115ceb94171b23de881aa4d469e P fc3b99856afe4cc3a86cc67240de20e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:58:05.355496 11024 raft_consensus.cc:729] T ea8cc115ceb94171b23de881aa4d469e P fc3b99856afe4cc3a86cc67240de20e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc3b99856afe4cc3a86cc67240de20e7, State: Initialized, Role: FOLLOWER
I20260811 18:58:05.355772 11028 tablet_bootstrap.cc:492] T ea8cc115ceb94171b23de881aa4d469e P fa32c49cd161420c979c48cd9fad3b05: Bootstrap starting.
I20260811 18:58:05.355764 11024 consensus_queue.cc:260] T ea8cc115ceb94171b23de881aa4d469e P fc3b99856afe4cc3a86cc67240de20e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc3b99856afe4cc3a86cc67240de20e7" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 35525 } } peers { permanent_uuid: "aaab81ed038149258391f548b07268af" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 37583 } } peers { permanent_uuid: "fa32c49cd161420c979c48cd9fad3b05" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 41101 } }
I20260811 18:58:05.359426 11030 tablet_bootstrap.cc:492] T ea8cc115ceb94171b23de881aa4d469e P aaab81ed038149258391f548b07268af: Bootstrap starting.
I20260811 18:58:05.358913 11028 tablet_bootstrap.cc:654] T ea8cc115ceb94171b23de881aa4d469e P fa32c49cd161420c979c48cd9fad3b05: Neither blocks nor log segments found. Creating new log.
I20260811 18:58:05.360267 11024 ts_tablet_manager.cc:1428] T ea8cc115ceb94171b23de881aa4d469e P fc3b99856afe4cc3a86cc67240de20e7: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 18:58:05.361174 10983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:05.360656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37199 (local address 127.29.112.129:45253)
0811 18:58:05.360899 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:05.361012 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:37199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
I20260811 18:58:05.361610 11030 tablet_bootstrap.cc:654] T ea8cc115ceb94171b23de881aa4d469e P aaab81ed038149258391f548b07268af: Neither blocks nor log segments found. Creating new log.
I20260811 18:58:05.364547 10342 heartbeater.cc:503] Master 127.29.112.188:44651 was elected leader, sending a full tablet report...
I20260811 18:58:05.365398 10983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:05.364510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37199 (local address 127.29.112.129:42123)
0811 18:58:05.365064 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:05.365183 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:37199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":396}
I20260811 18:58:05.369755 11030 tablet_bootstrap.cc:492] T ea8cc115ceb94171b23de881aa4d469e P aaab81ed038149258391f548b07268af: No bootstrap required, opened a new log
I20260811 18:58:05.369762 11028 tablet_bootstrap.cc:492] T ea8cc115ceb94171b23de881aa4d469e P fa32c49cd161420c979c48cd9fad3b05: No bootstrap required, opened a new log
I20260811 18:58:05.369906 11030 ts_tablet_manager.cc:1397] T ea8cc115ceb94171b23de881aa4d469e P aaab81ed038149258391f548b07268af: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.005s
I20260811 18:58:05.369939 11028 ts_tablet_manager.cc:1397] T ea8cc115ceb94171b23de881aa4d469e P fa32c49cd161420c979c48cd9fad3b05: Time spent bootstrapping tablet: real 0.014s user 0.011s sys 0.000s
I20260811 18:58:05.370628 11028 raft_consensus.cc:348] T ea8cc115ceb94171b23de881aa4d469e P fa32c49cd161420c979c48cd9fad3b05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc3b99856afe4cc3a86cc67240de20e7" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 35525 } } peers { permanent_uuid: "aaab81ed038149258391f548b07268af" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 37583 } } peers { permanent_uuid: "fa32c49cd161420c979c48cd9fad3b05" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 41101 } }
I20260811 18:58:05.370658 11030 raft_consensus.cc:348] T ea8cc115ceb94171b23de881aa4d469e P aaab81ed038149258391f548b07268af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc3b99856afe4cc3a86cc67240de20e7" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 35525 } } peers { permanent_uuid: "aaab81ed038149258391f548b07268af" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 37583 } } peers { permanent_uuid: "fa32c49cd161420c979c48cd9fad3b05" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 41101 } }
I20260811 18:58:05.370898 11028 raft_consensus.cc:374] T ea8cc115ceb94171b23de881aa4d469e P fa32c49cd161420c979c48cd9fad3b05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:58:05.370913 11030 raft_consensus.cc:374] T ea8cc115ceb94171b23de881aa4d469e P aaab81ed038149258391f548b07268af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:58:05.371016 11030 raft_consensus.cc:729] T ea8cc115ceb94171b23de881aa4d469e P aaab81ed038149258391f548b07268af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aaab81ed038149258391f548b07268af, State: Initialized, Role: FOLLOWER
I20260811 18:58:05.371027 11028 raft_consensus.cc:729] T ea8cc115ceb94171b23de881aa4d469e P fa32c49cd161420c979c48cd9fad3b05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa32c49cd161420c979c48cd9fad3b05, State: Initialized, Role: FOLLOWER
I20260811 18:58:05.371284 11028 consensus_queue.cc:260] T ea8cc115ceb94171b23de881aa4d469e P fa32c49cd161420c979c48cd9fad3b05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc3b99856afe4cc3a86cc67240de20e7" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 35525 } } peers { permanent_uuid: "aaab81ed038149258391f548b07268af" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 37583 } } peers { permanent_uuid: "fa32c49cd161420c979c48cd9fad3b05" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 41101 } }
I20260811 18:58:05.371266 11030 consensus_queue.cc:260] T ea8cc115ceb94171b23de881aa4d469e P aaab81ed038149258391f548b07268af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc3b99856afe4cc3a86cc67240de20e7" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 35525 } } peers { permanent_uuid: "aaab81ed038149258391f548b07268af" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 37583 } } peers { permanent_uuid: "fa32c49cd161420c979c48cd9fad3b05" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 41101 } }
I20260811 18:58:05.372192 11031 raft_consensus.cc:482] T ea8cc115ceb94171b23de881aa4d469e P fc3b99856afe4cc3a86cc67240de20e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:58:05.372397 11030 ts_tablet_manager.cc:1428] T ea8cc115ceb94171b23de881aa4d469e P aaab81ed038149258391f548b07268af: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260811 18:58:05.372408 11031 raft_consensus.cc:504] T ea8cc115ceb94171b23de881aa4d469e P fc3b99856afe4cc3a86cc67240de20e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc3b99856afe4cc3a86cc67240de20e7" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 35525 } } peers { permanent_uuid: "aaab81ed038149258391f548b07268af" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 37583 } } peers { permanent_uuid: "fa32c49cd161420c979c48cd9fad3b05" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 41101 } }
I20260811 18:58:05.373584 11028 ts_tablet_manager.cc:1428] T ea8cc115ceb94171b23de881aa4d469e P fa32c49cd161420c979c48cd9fad3b05: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260811 18:58:05.374194 10989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:05.373324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37199 (local address 127.29.112.130:50275)
0811 18:58:05.373958 (+ 634us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:05.374041 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:37199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":515}
I20260811 18:58:05.374418 11031 leader_election.cc:288] T ea8cc115ceb94171b23de881aa4d469e P fc3b99856afe4cc3a86cc67240de20e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aaab81ed038149258391f548b07268af (127.29.112.130:37583), fa32c49cd161420c979c48cd9fad3b05 (127.29.112.131:41101)
I20260811 18:58:05.374513 10986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:05.374100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37199 (local address 127.29.112.131:35053)
0811 18:58:05.374287 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:05.374386 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:37199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260811 18:58:05.375702 10986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:05.375162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37199 (local address 127.29.112.131:56119)
0811 18:58:05.375477 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:05.375562 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:37199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217}
I20260811 18:58:05.376724 10591 heartbeater.cc:503] Master 127.29.112.188:44651 was elected leader, sending a full tablet report...
I20260811 18:58:05.379069 10833 heartbeater.cc:503] Master 127.29.112.188:44651 was elected leader, sending a full tablet report...
I20260811 18:58:05.382648 11016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:05.375166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.129:38879 (local address 127.29.112.130:37583)
0811 18:58:05.375351 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:05.375357 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:05.375803 (+ 446us) server_negotiation.cc:407] Connection header received
0811 18:58:05.376017 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:05.376022 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:05.376090 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:05.376184 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:58:05.376805 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:05.377858 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:05.380436 (+ 2578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:05.381509 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:05.381699 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:05.381770 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:05.382220 (+ 450us) server_negotiation.cc:299] Negotiation successful
0811 18:58:05.382369 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 18:58:05.383325 11040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:05.374746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.130:37583 (local address 127.29.112.129:38879)
0811 18:58:05.375705 (+ 959us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:05.375718 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:05.375855 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:05.376214 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:05.376219 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:05.376243 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:58:05.376586 (+ 343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:05.376606 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:05.378085 (+ 1479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:05.378089 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:05.380208 (+ 2119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:05.380226 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:05.381721 (+ 1495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:05.381727 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:05.381983 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:05.382023 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 18:58:05.382815 (+ 792us) client_negotiation.cc:240] Negotiation successful
0811 18:58:05.382957 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":855,"mutex_wait_us":38,"thread_start_us":675,"threads_started":1}
I20260811 18:58:05.383910 10513 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea8cc115ceb94171b23de881aa4d469e" candidate_uuid: "fc3b99856afe4cc3a86cc67240de20e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aaab81ed038149258391f548b07268af" is_pre_election: true
I20260811 18:58:05.384104 10513 raft_consensus.cc:2454] T ea8cc115ceb94171b23de881aa4d469e P aaab81ed038149258391f548b07268af [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc3b99856afe4cc3a86cc67240de20e7 in term 0.
I20260811 18:58:05.385037 10989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:05.384646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37199 (local address 127.29.112.130:43401)
0811 18:58:05.384817 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:05.384904 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:37199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260811 18:58:05.386730 10166 leader_election.cc:302] T ea8cc115ceb94171b23de881aa4d469e P fc3b99856afe4cc3a86cc67240de20e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aaab81ed038149258391f548b07268af, fc3b99856afe4cc3a86cc67240de20e7; no voters:
I20260811 18:58:05.387421 11031 raft_consensus.cc:2781] T ea8cc115ceb94171b23de881aa4d469e P fc3b99856afe4cc3a86cc67240de20e7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:58:05.387509 11020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:05.376880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.129:59991 (local address 127.29.112.131:41101)
0811 18:58:05.377263 (+ 383us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:05.377269 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:05.377287 (+ 18us) server_negotiation.cc:407] Connection header received
0811 18:58:05.377347 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:05.377351 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:05.377415 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:05.377507 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:58:05.380585 (+ 3078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:05.381626 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:05.384196 (+ 2570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:05.385639 (+ 1443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:05.385909 (+ 270us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:05.386018 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:05.386805 (+ 787us) server_negotiation.cc:299] Negotiation successful
0811 18:58:05.387226 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262}
I20260811 18:58:05.387418 10983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:05.374670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.131:41101 (local address 127.29.112.129:59991)
0811 18:58:05.375173 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:05.375192 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:05.375311 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:05.380053 (+ 4742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:05.380063 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:05.380090 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:58:05.380429 (+ 339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:05.380438 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:05.381927 (+ 1489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:05.381935 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:05.383940 (+ 2005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:05.383952 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:05.385993 (+ 2041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:05.386001 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:05.386477 (+ 476us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:05.386521 (+ 44us) client_negotiation.cc:769] Sending connection context
0811 18:58:05.386765 (+ 244us) client_negotiation.cc:240] Negotiation successful
0811 18:58:05.387013 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":395,"mutex_wait_us":2}
I20260811 18:58:05.387610 11031 raft_consensus.cc:482] T ea8cc115ceb94171b23de881aa4d469e P fc3b99856afe4cc3a86cc67240de20e7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:58:05.387727 11031 raft_consensus.cc:3037] T ea8cc115ceb94171b23de881aa4d469e P fc3b99856afe4cc3a86cc67240de20e7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:58:05.388746 10756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea8cc115ceb94171b23de881aa4d469e" candidate_uuid: "fc3b99856afe4cc3a86cc67240de20e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa32c49cd161420c979c48cd9fad3b05" is_pre_election: true
I20260811 18:58:05.389088 10756 raft_consensus.cc:2454] T ea8cc115ceb94171b23de881aa4d469e P fa32c49cd161420c979c48cd9fad3b05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc3b99856afe4cc3a86cc67240de20e7 in term 0.
I20260811 18:58:05.390352 11031 raft_consensus.cc:504] T ea8cc115ceb94171b23de881aa4d469e P fc3b99856afe4cc3a86cc67240de20e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc3b99856afe4cc3a86cc67240de20e7" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 35525 } } peers { permanent_uuid: "aaab81ed038149258391f548b07268af" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 37583 } } peers { permanent_uuid: "fa32c49cd161420c979c48cd9fad3b05" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 41101 } }
I20260811 18:58:05.391067 11031 leader_election.cc:288] T ea8cc115ceb94171b23de881aa4d469e P fc3b99856afe4cc3a86cc67240de20e7 [CANDIDATE]: Term 1 election: Requested vote from peers aaab81ed038149258391f548b07268af (127.29.112.130:37583), fa32c49cd161420c979c48cd9fad3b05 (127.29.112.131:41101)
I20260811 18:58:05.391551 10513 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea8cc115ceb94171b23de881aa4d469e" candidate_uuid: "fc3b99856afe4cc3a86cc67240de20e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aaab81ed038149258391f548b07268af"
I20260811 18:58:05.391559 10756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea8cc115ceb94171b23de881aa4d469e" candidate_uuid: "fc3b99856afe4cc3a86cc67240de20e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa32c49cd161420c979c48cd9fad3b05"
I20260811 18:58:05.391755 10513 raft_consensus.cc:3037] T ea8cc115ceb94171b23de881aa4d469e P aaab81ed038149258391f548b07268af [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:58:05.391789 10756 raft_consensus.cc:3037] T ea8cc115ceb94171b23de881aa4d469e P fa32c49cd161420c979c48cd9fad3b05 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:58:05.394266 10756 raft_consensus.cc:2454] T ea8cc115ceb94171b23de881aa4d469e P fa32c49cd161420c979c48cd9fad3b05 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc3b99856afe4cc3a86cc67240de20e7 in term 1.
I20260811 18:58:05.394482 10513 raft_consensus.cc:2454] T ea8cc115ceb94171b23de881aa4d469e P aaab81ed038149258391f548b07268af [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc3b99856afe4cc3a86cc67240de20e7 in term 1.
I20260811 18:58:05.394990 10166 leader_election.cc:302] T ea8cc115ceb94171b23de881aa4d469e P fc3b99856afe4cc3a86cc67240de20e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aaab81ed038149258391f548b07268af, fc3b99856afe4cc3a86cc67240de20e7; no voters:
I20260811 18:58:05.395315 11031 raft_consensus.cc:2781] T ea8cc115ceb94171b23de881aa4d469e P fc3b99856afe4cc3a86cc67240de20e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:58:05.395896 11031 raft_consensus.cc:686] T ea8cc115ceb94171b23de881aa4d469e P fc3b99856afe4cc3a86cc67240de20e7 [term 1 LEADER]: Becoming Leader. State: Replica: fc3b99856afe4cc3a86cc67240de20e7, State: Running, Role: LEADER
I20260811 18:58:05.396404 11031 consensus_queue.cc:237] T ea8cc115ceb94171b23de881aa4d469e P fc3b99856afe4cc3a86cc67240de20e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc3b99856afe4cc3a86cc67240de20e7" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 35525 } } peers { permanent_uuid: "aaab81ed038149258391f548b07268af" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 37583 } } peers { permanent_uuid: "fa32c49cd161420c979c48cd9fad3b05" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 41101 } }
I20260811 18:58:05.397315 10983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:05.396797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37199 (local address 127.29.112.129:35093)
0811 18:58:05.396993 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:05.397082 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:37199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260811 18:58:05.400045 10037 catalog_manager.cc:5495] T ea8cc115ceb94171b23de881aa4d469e P fc3b99856afe4cc3a86cc67240de20e7 reported cstate change: term changed from 0 to 1, leader changed from <none> to fc3b99856afe4cc3a86cc67240de20e7 (127.29.112.129). New cstate: current_term: 1 leader_uuid: "fc3b99856afe4cc3a86cc67240de20e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc3b99856afe4cc3a86cc67240de20e7" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 35525 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aaab81ed038149258391f548b07268af" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 37583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa32c49cd161420c979c48cd9fad3b05" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 41101 } health_report { overall_health: UNKNOWN } } }
I20260811 18:58:05.430769 11013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:05.418472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35814 (local address 127.29.112.129:35525)
0811 18:58:05.418714 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:05.418722 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:05.418742 (+ 20us) server_negotiation.cc:407] Connection header received
0811 18:58:05.419000 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:05.419004 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:05.419092 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:05.419251 (+ 159us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:58:05.419627 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:05.420725 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:05.422027 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:05.422604 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:05.422742 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:05.423147 (+ 405us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:58:05.423778 (+ 631us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:58:05.423931 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:05.429448 (+ 5517us) server_negotiation.cc:299] Negotiation successful
0811 18:58:05.429577 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260811 18:58:05.437588 10513 raft_consensus.cc:1261] T ea8cc115ceb94171b23de881aa4d469e P aaab81ed038149258391f548b07268af [term 1 FOLLOWER]: Refusing update from remote peer fc3b99856afe4cc3a86cc67240de20e7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:58:05.437851 10756 raft_consensus.cc:1261] T ea8cc115ceb94171b23de881aa4d469e P fa32c49cd161420c979c48cd9fad3b05 [term 1 FOLLOWER]: Refusing update from remote peer fc3b99856afe4cc3a86cc67240de20e7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:58:05.438580 11041 consensus_queue.cc:1035] T ea8cc115ceb94171b23de881aa4d469e P fc3b99856afe4cc3a86cc67240de20e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aaab81ed038149258391f548b07268af" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 37583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:58:05.439128 11031 consensus_queue.cc:1035] T ea8cc115ceb94171b23de881aa4d469e P fc3b99856afe4cc3a86cc67240de20e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa32c49cd161420c979c48cd9fad3b05" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 41101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:58:05.441843 10986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:05.441355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37199 (local address 127.29.112.131:55423)
0811 18:58:05.441590 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:05.441682 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:37199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118}
I20260811 18:58:05.442039 10989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:05.441573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37199 (local address 127.29.112.130:45817)
0811 18:58:05.441780 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:05.441870 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:37199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117}
I20260811 18:58:05.452546 11048 mvcc.cc:205] Tried to move back new op lower bound from 7317400311544229888 to 7317400311387521024. Current Snapshot: MvccSnapshot[applied={T|T < 7317400311544229888}]
I20260811 18:58:05.454295 10989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:05.453612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37199 (local address 127.29.112.130:50663)
0811 18:58:05.453964 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:05.454074 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:37199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210}
I20260811 18:58:05.465973 11049 mvcc.cc:205] Tried to move back new op lower bound from 7317400311544229888 to 7317400311387521024. Current Snapshot: MvccSnapshot[applied={T|T < 7317400311544229888}]
I20260811 18:58:05.466938 10986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:05.466231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37199 (local address 127.29.112.131:35219)
0811 18:58:05.466568 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:05.466691 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:37199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157}
I20260811 18:58:05.469424 10983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:05.468942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37199 (local address 127.29.112.129:56351)
0811 18:58:05.469162 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:05.469258 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:37199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119}
I20260811 18:58:05.479053 10983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:05.478481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37199 (local address 127.29.112.129:48077)
0811 18:58:05.478766 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:05.478873 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:37199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152}
I20260811 18:58:05.485540 10037 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47648:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:58:05.485971 10037 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47648:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:58:05.493438 10037 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4198b17e42c94aac90009c1bc0970564: Sending DeleteTablet for 3 replicas of tablet ea8cc115ceb94171b23de881aa4d469e
I20260811 18:58:05.494553 10229 tablet_service.cc:1514] Processing DeleteTablet for tablet ea8cc115ceb94171b23de881aa4d469e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:58:05 UTC) from {username='oryx', principal='oryx/127.29.112.188@KRBTEST.COM'} at 127.0.0.1:35812
I20260811 18:58:05.494871 10480 tablet_service.cc:1514] Processing DeleteTablet for tablet ea8cc115ceb94171b23de881aa4d469e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:58:05 UTC) from {username='oryx', principal='oryx/127.29.112.188@KRBTEST.COM'} at 127.0.0.1:60586
I20260811 18:58:05.495599 11060 tablet_replica.cc:330] T ea8cc115ceb94171b23de881aa4d469e P aaab81ed038149258391f548b07268af: stopping tablet replica
I20260811 18:58:05.495844 11060 raft_consensus.cc:2229] T ea8cc115ceb94171b23de881aa4d469e P aaab81ed038149258391f548b07268af [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:58:05.496196 11060 raft_consensus.cc:2258] T ea8cc115ceb94171b23de881aa4d469e P aaab81ed038149258391f548b07268af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:58:05.496519 10719 tablet_service.cc:1514] Processing DeleteTablet for tablet ea8cc115ceb94171b23de881aa4d469e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:58:05 UTC) from {username='oryx', principal='oryx/127.29.112.188@KRBTEST.COM'} at 127.0.0.1:53292
I20260811 18:58:05.497288 11060 ts_tablet_manager.cc:1905] T ea8cc115ceb94171b23de881aa4d469e P aaab81ed038149258391f548b07268af: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:58:05.502112 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 9993
I20260811 18:58:05.506191 11060 ts_tablet_manager.cc:1918] T ea8cc115ceb94171b23de881aa4d469e P aaab81ed038149258391f548b07268af: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:58:05.506364 11060 log.cc:1196] T ea8cc115ceb94171b23de881aa4d469e P aaab81ed038149258391f548b07268af: Deleting WAL directory at /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/ts-1/wal/wals/ea8cc115ceb94171b23de881aa4d469e
I20260811 18:58:05.506980 11060 ts_tablet_manager.cc:1939] T ea8cc115ceb94171b23de881aa4d469e P aaab81ed038149258391f548b07268af: Deleting consensus metadata
I20260811 18:58:05.508553 10022 catalog_manager.cc:4841] TS aaab81ed038149258391f548b07268af (127.29.112.130:37583): tablet ea8cc115ceb94171b23de881aa4d469e (table test-table [id=e076620853a34ea5b99f3fc0802cde8c]) successfully deleted
I20260811 18:58:05.524037 11061 tablet_replica.cc:330] T ea8cc115ceb94171b23de881aa4d469e P fa32c49cd161420c979c48cd9fad3b05: stopping tablet replica
I20260811 18:58:05.524279 11061 raft_consensus.cc:2229] T ea8cc115ceb94171b23de881aa4d469e P fa32c49cd161420c979c48cd9fad3b05 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:58:05.524612 11061 raft_consensus.cc:2258] T ea8cc115ceb94171b23de881aa4d469e P fa32c49cd161420c979c48cd9fad3b05 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:58:05.524706 11059 tablet_replica.cc:330] T ea8cc115ceb94171b23de881aa4d469e P fc3b99856afe4cc3a86cc67240de20e7: stopping tablet replica
I20260811 18:58:05.524933 11059 raft_consensus.cc:2229] T ea8cc115ceb94171b23de881aa4d469e P fc3b99856afe4cc3a86cc67240de20e7 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:58:05.525753 11059 raft_consensus.cc:2258] T ea8cc115ceb94171b23de881aa4d469e P fc3b99856afe4cc3a86cc67240de20e7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:58:05.526515 11061 ts_tablet_manager.cc:1905] T ea8cc115ceb94171b23de881aa4d469e P fa32c49cd161420c979c48cd9fad3b05: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:58:05.526978 11059 ts_tablet_manager.cc:1905] T ea8cc115ceb94171b23de881aa4d469e P fc3b99856afe4cc3a86cc67240de20e7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:58:05.535977 11059 ts_tablet_manager.cc:1918] T ea8cc115ceb94171b23de881aa4d469e P fc3b99856afe4cc3a86cc67240de20e7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:58:05.536146 11059 log.cc:1196] T ea8cc115ceb94171b23de881aa4d469e P fc3b99856afe4cc3a86cc67240de20e7: Deleting WAL directory at /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/ts-0/wal/wals/ea8cc115ceb94171b23de881aa4d469e
I20260811 18:58:05.536703 11059 ts_tablet_manager.cc:1939] T ea8cc115ceb94171b23de881aa4d469e P fc3b99856afe4cc3a86cc67240de20e7: Deleting consensus metadata
I20260811 18:58:05.536859 11061 ts_tablet_manager.cc:1918] T ea8cc115ceb94171b23de881aa4d469e P fa32c49cd161420c979c48cd9fad3b05: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:58:05.537088 11061 log.cc:1196] T ea8cc115ceb94171b23de881aa4d469e P fa32c49cd161420c979c48cd9fad3b05: Deleting WAL directory at /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474615205622-30146-0/minicluster-data/ts-2/wal/wals/ea8cc115ceb94171b23de881aa4d469e
I20260811 18:58:05.537671 11061 ts_tablet_manager.cc:1939] T ea8cc115ceb94171b23de881aa4d469e P fa32c49cd161420c979c48cd9fad3b05: Deleting consensus metadata
I20260811 18:58:05.538057 10022 catalog_manager.cc:4841] TS fc3b99856afe4cc3a86cc67240de20e7 (127.29.112.129:35525): tablet ea8cc115ceb94171b23de881aa4d469e (table test-table [id=e076620853a34ea5b99f3fc0802cde8c]) successfully deleted
I20260811 18:58:05.539140 10018 catalog_manager.cc:4841] TS fa32c49cd161420c979c48cd9fad3b05 (127.29.112.131:41101): tablet ea8cc115ceb94171b23de881aa4d469e (table test-table [id=e076620853a34ea5b99f3fc0802cde8c]) successfully deleted
I20260811 18:58:05.547632 11013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:05.503371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35822 (local address 127.29.112.129:35525)
0811 18:58:05.503554 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:05.503562 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:05.525348 (+ 21786us) server_negotiation.cc:407] Connection header received
0811 18:58:05.525579 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:05.525584 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:05.525749 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:05.525867 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:05.527990 (+ 2123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:05.529228 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:05.532214 (+ 2986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:05.532992 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:05.533139 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:05.538799 (+ 5660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:58:05.538819 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:58:05.538849 (+ 30us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:58:05.538906 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:58:05.543936 (+ 5030us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:05.544863 (+ 927us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:05.544876 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:05.544882 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:05.545088 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:05.545562 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:05.545570 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:05.545575 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:05.546119 (+ 544us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:58:05.546499 (+ 380us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:05.546779 (+ 280us) server_negotiation.cc:299] Negotiation successful
0811 18:58:05.547053 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 18:58:05.548586 10335 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:58:05.651451 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 9993
I20260811 18:58:05.676940 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 10363
I20260811 18:58:05.692128 11016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:05.677899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60588 (local address 127.29.112.130:37583)
0811 18:58:05.678180 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:05.678185 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:05.678202 (+ 17us) server_negotiation.cc:407] Connection header received
0811 18:58:05.678260 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:05.678264 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:05.678335 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:05.678426 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:05.679753 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:05.680700 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:05.682068 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:05.682478 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:05.682678 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:05.685303 (+ 2625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:58:05.685316 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:58:05.685333 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:58:05.685384 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:58:05.689024 (+ 3640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:05.689788 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:05.689794 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:05.689797 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:05.689992 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:05.690362 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:05.690365 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:05.690367 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:05.690970 (+ 603us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:58:05.691310 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:05.691484 (+ 174us) server_negotiation.cc:299] Negotiation successful
0811 18:58:05.691667 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159}
I20260811 18:58:05.692934 10583 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:58:05.787739 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 10363
I20260811 18:58:05.812669 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 10611
I20260811 18:58:05.826314 11020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:05.813483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53302 (local address 127.29.112.131:41101)
0811 18:58:05.813701 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:05.813705 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:05.813724 (+ 19us) server_negotiation.cc:407] Connection header received
0811 18:58:05.813880 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:05.813884 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:05.813961 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:05.814046 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:05.815596 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:05.816342 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:05.817579 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:05.817942 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:05.818038 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:05.820554 (+ 2516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:58:05.820566 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:58:05.820581 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:58:05.820610 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:58:05.823318 (+ 2708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:05.823954 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:05.823960 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:05.823962 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:05.824088 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:05.824517 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:05.824520 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:05.824522 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:05.824893 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:58:05.825128 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:05.825653 (+ 525us) server_negotiation.cc:299] Negotiation successful
0811 18:58:05.825814 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260811 18:58:05.827004 10821 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:58:05.829802 10992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:05.829428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37199 (local address 127.0.0.1:33048)
0811 18:58:05.829584 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:05.829675 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:37199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260811 18:58:05.845147 10992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:05.844685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37199 (local address 127.0.0.1:33060)
0811 18:58:05.844858 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:05.844977 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:37199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260811 18:58:05.947337 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 10611
I20260811 18:58:05.976410 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 9454
I20260811 18:58:05.977645 9742 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:58:06.098302 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 9454
I20260811 18:58:06.131739 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 9619
I20260811 18:58:06.133299 10084 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:58:06.258381 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 9619
2026-08-11T18:58:06Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (11990 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:58:06 dist-test-slave-h6n0 krb5kdc[11077](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:58:06 dist-test-slave-h6n0 krb5kdc[11077](info): set up 2 sockets
Aug 11 18:58:06 dist-test-slave-h6n0 krb5kdc[11077](info): commencing operation
krb5kdc: starting...
W20260811 18:58:07.664361 30146 mini_kdc.cc:121] Time spent starting KDC: real 1.345s user 0.000s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/krb5kdc/test-user.keytab.
Aug 11 18:58:07 dist-test-slave-h6n0 krb5kdc[11077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474687, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:58:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:58:07Z Disabled control of system clock
WARNING: no policy specified for oryx/127.29.112.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.112.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.112.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/krb5kdc/oryx_127.29.112.190.keytab.
Entry for principal oryx/127.29.112.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/krb5kdc/oryx_127.29.112.190.keytab.
I20260811 18:58:07.938861 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.190:41859
--webserver_interface=127.29.112.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/krb5kdc/oryx_127.29.112.190.keytab
--principal=oryx/127.29.112.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:36951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.190:41859
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:58:08.301400 11099 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:58:08.301746 11099 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:58:08.301903 11099 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:58:08.309218 11099 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:58:08.309365 11099 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 18:58:08.309454 11099 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:58:08.309581 11099 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:58:08.309667 11099 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:58:08.309762 11099 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:58:08.320087 11099 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:36951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.112.190:41859
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/krb5kdc/oryx_127.29.112.190.keytab
--principal=oryx/127.29.112.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.190:41859
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.112.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:58:08.321455 11099 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:58:08.322819 11099 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:58:08.335515 11106 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:08.335481 11112 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:08.335685 11108 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:58:08.337317 11099 server_base.cc:1034] running on GCE node
I20260811 18:58:08.338200 11099 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:58:08.339831 11099 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:58:08.341154 11099 hybrid_clock.cc:648] HybridClock initialized: now 1786474688341086 us; error 56 us; skew 500 ppm
I20260811 18:58:08.341485 11099 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:58:08 dist-test-slave-h6n0 krb5kdc[11077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474688, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.112.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:58:08.350394 11099 init.cc:377] Logged in from keytab as oryx/127.29.112.190@KRBTEST.COM (short username oryx)
I20260811 18:58:08.352092 11099 webserver.cc:460] Webserver started at http://127.29.112.190:39669/ using document root <none> and password file <none>
I20260811 18:58:08.352869 11099 fs_manager.cc:359] Metadata directory not provided
I20260811 18:58:08.353004 11099 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:58:08.353369 11099 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:58:08.356715 11099 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/minicluster-data/master-0/data/instance:
uuid: "392e71a7a404425489017441bdfbad94"
format_stamp: "Formatted at 2026-08-11 18:58:08 on dist-test-slave-h6n0"
server_key: "04292579708222aeb7b93fba22569418"
server_key_iv: "4863e021d68e4abbe1fb6e41507b3442"
server_key_version: "encryptionkey@0"
I20260811 18:58:08.357599 11099 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/minicluster-data/master-0/wal/instance:
uuid: "392e71a7a404425489017441bdfbad94"
format_stamp: "Formatted at 2026-08-11 18:58:08 on dist-test-slave-h6n0"
server_key: "04292579708222aeb7b93fba22569418"
server_key_iv: "4863e021d68e4abbe1fb6e41507b3442"
server_key_version: "encryptionkey@0"
I20260811 18:58:08.363332 11099 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260811 18:58:08.367187 11134 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:08.368973 11099 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 18:58:08.369366 11099 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/minicluster-data/master-0/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/minicluster-data/master-0/wal
uuid: "392e71a7a404425489017441bdfbad94"
format_stamp: "Formatted at 2026-08-11 18:58:08 on dist-test-slave-h6n0"
server_key: "04292579708222aeb7b93fba22569418"
server_key_iv: "4863e021d68e4abbe1fb6e41507b3442"
server_key_version: "encryptionkey@0"
I20260811 18:58:08.369654 11099 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:58:08.389149 11099 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:58:08.393812 11099 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:58:08.394089 11099 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:58:08.421712 11099 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.190:41859
I20260811 18:58:08.421902 11230 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.190:41859 every 8 connection(s)
I20260811 18:58:08.423207 11099 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
I20260811 18:58:08.427731 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 11099
I20260811 18:58:08.428251 11232 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:58:08.428282 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/minicluster-data/master-0/wal/instance
I20260811 18:58:08.429177 30146 external_mini_cluster.cc:1428] Setting key 2e030f535aa808849d931590087cbe32
I20260811 18:58:08.443737 11232 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 392e71a7a404425489017441bdfbad94: Bootstrap starting.
Aug 11 18:58:08 dist-test-slave-h6n0 krb5kdc[11077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474687, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM
I20260811 18:58:08.447482 11232 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 392e71a7a404425489017441bdfbad94: Neither blocks nor log segments found. Creating new log.
I20260811 18:58:08.448622 11232 log.cc:824] T 00000000000000000000000000000000 P 392e71a7a404425489017441bdfbad94: Log is configured to *not* fsync() on all Append() calls
I20260811 18:58:08.451792 11232 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 392e71a7a404425489017441bdfbad94: No bootstrap required, opened a new log
I20260811 18:58:08.455987 11236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:08.431677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36276 (local address 127.29.112.190:41859)
0811 18:58:08.432583 (+ 906us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:08.432596 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:08.432639 (+ 43us) server_negotiation.cc:407] Connection header received
0811 18:58:08.435583 (+ 2944us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:08.435612 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:08.436050 (+ 438us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:08.436490 (+ 440us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:08.438217 (+ 1727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:08.439638 (+ 1421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:08.440955 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:08.441768 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:08.441982 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:08.447110 (+ 5128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:58:08.447119 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:58:08.447133 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:58:08.447174 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:58:08.451137 (+ 3963us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:08.451868 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:08.451881 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:08.451886 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:08.452514 (+ 628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:08.453002 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:08.453008 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:08.453011 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:08.453597 (+ 586us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:58:08.453914 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:08.454430 (+ 516us) server_negotiation.cc:299] Negotiation successful
0811 18:58:08.454946 (+ 516us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":673,"thread_start_us":404,"threads_started":1}
I20260811 18:58:08.462975 11232 raft_consensus.cc:348] T 00000000000000000000000000000000 P 392e71a7a404425489017441bdfbad94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "392e71a7a404425489017441bdfbad94" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 41859 } }
I20260811 18:58:08.463431 11232 raft_consensus.cc:374] T 00000000000000000000000000000000 P 392e71a7a404425489017441bdfbad94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:58:08.463549 11232 raft_consensus.cc:729] T 00000000000000000000000000000000 P 392e71a7a404425489017441bdfbad94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 392e71a7a404425489017441bdfbad94, State: Initialized, Role: FOLLOWER
I20260811 18:58:08.464140 11232 consensus_queue.cc:260] T 00000000000000000000000000000000 P 392e71a7a404425489017441bdfbad94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "392e71a7a404425489017441bdfbad94" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 41859 } }
I20260811 18:58:08.464412 11232 raft_consensus.cc:388] T 00000000000000000000000000000000 P 392e71a7a404425489017441bdfbad94 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:58:08.464560 11232 raft_consensus.cc:482] T 00000000000000000000000000000000 P 392e71a7a404425489017441bdfbad94 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:58:08.464749 11232 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 392e71a7a404425489017441bdfbad94 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:58:08.467092 11232 raft_consensus.cc:504] T 00000000000000000000000000000000 P 392e71a7a404425489017441bdfbad94 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "392e71a7a404425489017441bdfbad94" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 41859 } }
I20260811 18:58:08.467650 11232 leader_election.cc:302] T 00000000000000000000000000000000 P 392e71a7a404425489017441bdfbad94 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 392e71a7a404425489017441bdfbad94; no voters:
I20260811 18:58:08.468259 11232 leader_election.cc:288] T 00000000000000000000000000000000 P 392e71a7a404425489017441bdfbad94 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:58:08.468454 11238 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 392e71a7a404425489017441bdfbad94 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:58:08.469309 11238 raft_consensus.cc:686] T 00000000000000000000000000000000 P 392e71a7a404425489017441bdfbad94 [term 1 LEADER]: Becoming Leader. State: Replica: 392e71a7a404425489017441bdfbad94, State: Running, Role: LEADER
I20260811 18:58:08.469992 11238 consensus_queue.cc:237] T 00000000000000000000000000000000 P 392e71a7a404425489017441bdfbad94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "392e71a7a404425489017441bdfbad94" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 41859 } }
I20260811 18:58:08.470621 11232 sys_catalog.cc:564] T 00000000000000000000000000000000 P 392e71a7a404425489017441bdfbad94 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:58:08.473011 11240 sys_catalog.cc:455] T 00000000000000000000000000000000 P 392e71a7a404425489017441bdfbad94 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 392e71a7a404425489017441bdfbad94. Latest consensus state: current_term: 1 leader_uuid: "392e71a7a404425489017441bdfbad94" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "392e71a7a404425489017441bdfbad94" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 41859 } } }
I20260811 18:58:08.473459 11240 sys_catalog.cc:458] T 00000000000000000000000000000000 P 392e71a7a404425489017441bdfbad94 [sys.catalog]: This master's current role is: LEADER
I20260811 18:58:08.474000 11239 sys_catalog.cc:455] T 00000000000000000000000000000000 P 392e71a7a404425489017441bdfbad94 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "392e71a7a404425489017441bdfbad94" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "392e71a7a404425489017441bdfbad94" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 41859 } } }
I20260811 18:58:08.474318 11239 sys_catalog.cc:458] T 00000000000000000000000000000000 P 392e71a7a404425489017441bdfbad94 [sys.catalog]: This master's current role is: LEADER
I20260811 18:58:08.475296 11256 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:58:08.480309 11256 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:58:08.491408 11256 catalog_manager.cc:1347] Generated new cluster ID: 539c58916bb141e3bf486e7161d98a07
I20260811 18:58:08.491566 11256 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:58:08.518576 11256 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:58:08.519935 11256 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:58:08.526185 11256 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 392e71a7a404425489017441bdfbad94: Generated new TSK 0
I20260811 18:58:08.526988 11256 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.29.112.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.112.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.112.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/krb5kdc/oryx_127.29.112.129.keytab.
Entry for principal oryx/127.29.112.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/krb5kdc/oryx_127.29.112.129.keytab.
I20260811 18:58:08.596047 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.129:0
--local_ip_for_outbound_sockets=127.29.112.129
--webserver_interface=127.29.112.129
--webserver_port=0
--tserver_master_addrs=127.29.112.190:41859
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/krb5kdc/oryx_127.29.112.129.keytab
--principal=oryx/127.29.112.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:36951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:58:08.952463 11274 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:58:08.952785 11274 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:58:08.952979 11274 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:58:08.960388 11274 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:58:08.960561 11274 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:58:08.960705 11274 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 18:58:08.960842 11274 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.129
I20260811 18:58:08.970726 11274 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:36951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/krb5kdc/oryx_127.29.112.129.keytab
--principal=oryx/127.29.112.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.112.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.112.190:41859
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.129
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:58:08.972110 11274 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:58:08.973608 11274 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:58:08.987502 11282 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:08.987534 11285 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:08.987568 11281 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:58:08.988509 11274 server_base.cc:1034] running on GCE node
I20260811 18:58:08.989125 11274 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:58:08.990031 11274 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:58:08.991312 11274 hybrid_clock.cc:648] HybridClock initialized: now 1786474688991232 us; error 53 us; skew 500 ppm
I20260811 18:58:08.991664 11274 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:58:08 dist-test-slave-h6n0 krb5kdc[11077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474688, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.112.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:58:08.995011 11274 init.cc:377] Logged in from keytab as oryx/127.29.112.129@KRBTEST.COM (short username oryx)
I20260811 18:58:08.996538 11274 webserver.cc:460] Webserver started at http://127.29.112.129:43959/ using document root <none> and password file <none>
I20260811 18:58:08.997192 11274 fs_manager.cc:359] Metadata directory not provided
I20260811 18:58:08.997279 11274 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:58:08.997581 11274 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:58:09.000474 11274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/minicluster-data/ts-0/data/instance:
uuid: "c2d72e507d484000b6768f9a7f376ffb"
format_stamp: "Formatted at 2026-08-11 18:58:08 on dist-test-slave-h6n0"
server_key: "1df891d1dcd52aebf0ddcce1013dfffc"
server_key_iv: "ea12bcd3f45a474ec34b1e778ae826ce"
server_key_version: "encryptionkey@0"
I20260811 18:58:09.001238 11274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/minicluster-data/ts-0/wal/instance:
uuid: "c2d72e507d484000b6768f9a7f376ffb"
format_stamp: "Formatted at 2026-08-11 18:58:08 on dist-test-slave-h6n0"
server_key: "1df891d1dcd52aebf0ddcce1013dfffc"
server_key_iv: "ea12bcd3f45a474ec34b1e778ae826ce"
server_key_version: "encryptionkey@0"
I20260811 18:58:09.006192 11274 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 18:58:09.010756 11297 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:09.012020 11274 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:58:09.012190 11274 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/minicluster-data/ts-0/wal
uuid: "c2d72e507d484000b6768f9a7f376ffb"
format_stamp: "Formatted at 2026-08-11 18:58:08 on dist-test-slave-h6n0"
server_key: "1df891d1dcd52aebf0ddcce1013dfffc"
server_key_iv: "ea12bcd3f45a474ec34b1e778ae826ce"
server_key_version: "encryptionkey@0"
I20260811 18:58:09.012413 11274 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:58:09.034890 11274 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:58:09.038820 11274 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:58:09.039381 11274 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:58:09.042809 11274 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:58:09.042918 11274 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:09.043073 11274 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:58:09.043206 11274 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
Aug 11 18:58:09 dist-test-slave-h6n0 krb5kdc[11077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474688, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.112.129@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM
I20260811 18:58:09.067685 11317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:09.043378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.129:41879 (local address 127.29.112.190:41859)
0811 18:58:09.045861 (+ 2483us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:09.045868 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:09.045893 (+ 25us) server_negotiation.cc:407] Connection header received
0811 18:58:09.047726 (+ 1833us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:09.047732 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:09.047829 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:09.047968 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:09.050945 (+ 2977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:09.051921 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:09.054445 (+ 2524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:09.054942 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:09.055097 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:09.059424 (+ 4327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:58:09.059431 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:58:09.059436 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:58:09.059478 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:58:09.063125 (+ 3647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:09.064277 (+ 1152us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:09.064284 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:09.064289 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:09.064434 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:09.065099 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:09.065102 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:09.065105 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:09.065361 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:58:09.065524 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:09.066803 (+ 1279us) server_negotiation.cc:299] Negotiation successful
0811 18:58:09.067133 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2340,"thread_start_us":2238,"threads_started":1}
I20260811 18:58:09.068681 11319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:09.044147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:41859 (local address 127.29.112.129:41879)
0811 18:58:09.045231 (+ 1084us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:09.045300 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:09.047444 (+ 2144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:09.048432 (+ 988us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:09.048449 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:09.049482 (+ 1033us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:58:09.050629 (+ 1147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:09.050660 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:09.052176 (+ 1516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:09.052187 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:09.054274 (+ 2087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:09.054286 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:09.055200 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:09.055210 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:09.055440 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:09.056130 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:58:09.056157 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:58:09.059199 (+ 3042us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:58:09.063283 (+ 4084us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:58:09.063293 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:58:09.063316 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:58:09.064082 (+ 766us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:58:09.064564 (+ 482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:58:09.064569 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:58:09.064573 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:58:09.064926 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:58:09.065509 (+ 583us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:58:09.065516 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:58:09.066086 (+ 570us) client_negotiation.cc:769] Sending connection context
0811 18:58:09.066442 (+ 356us) client_negotiation.cc:240] Negotiation successful
0811 18:58:09.066971 (+ 529us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":704,"thread_start_us":427,"threads_started":1}
I20260811 18:58:09.091790 11420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:09.076374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:41859 (local address 127.29.112.129:58101)
0811 18:58:09.077238 (+ 864us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:09.077265 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:09.077456 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:09.077797 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:09.077805 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:09.078632 (+ 827us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:58:09.079258 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:09.079277 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:09.080505 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:09.080514 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:09.082388 (+ 1874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:09.082400 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:09.083376 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:09.083384 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:09.083550 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:09.084315 (+ 765us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:58:09.084343 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:58:09.085096 (+ 753us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:58:09.088670 (+ 3574us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:58:09.088675 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:58:09.088679 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:58:09.089285 (+ 606us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:58:09.089703 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:58:09.089708 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:58:09.089712 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:58:09.089857 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:58:09.090421 (+ 564us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:58:09.090426 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:58:09.090689 (+ 263us) client_negotiation.cc:769] Sending connection context
0811 18:58:09.090885 (+ 196us) client_negotiation.cc:240] Negotiation successful
0811 18:58:09.091193 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":653,"thread_start_us":600,"threads_started":1}
I20260811 18:58:09.092676 11317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:09.076912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.129:58101 (local address 127.29.112.190:41859)
0811 18:58:09.077088 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:09.077094 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:09.077371 (+ 277us) server_negotiation.cc:407] Connection header received
0811 18:58:09.077579 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:09.077583 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:09.077646 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:09.077727 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:09.079439 (+ 1712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:09.080358 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:09.082576 (+ 2218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:09.083177 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:09.083310 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:09.085295 (+ 1985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:58:09.085306 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:58:09.085310 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:58:09.085341 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:58:09.088478 (+ 3137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:09.089435 (+ 957us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:09.089441 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:09.089445 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:09.089582 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:09.089979 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:09.089986 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:09.089989 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:09.090258 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:58:09.090449 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:09.092277 (+ 1828us) server_negotiation.cc:299] Negotiation successful
0811 18:58:09.092449 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 18:58:09.100883 11274 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.129:37641
I20260811 18:58:09.100910 11501 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.129:37641 every 8 connection(s)
I20260811 18:58:09.102344 11274 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/minicluster-data/ts-0/data/info.pb
I20260811 18:58:09.104753 11503 heartbeater.cc:347] Connected to a master server at 127.29.112.190:41859
I20260811 18:58:09.105033 11503 heartbeater.cc:464] Registering TS with master...
I20260811 18:58:09.105119 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 11274
I20260811 18:58:09.105347 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/minicluster-data/ts-0/wal/instance
I20260811 18:58:09.105852 11503 heartbeater.cc:511] Master 127.29.112.190:41859 requested a full tablet report, sending...
I20260811 18:58:09.106232 30146 external_mini_cluster.cc:1428] Setting key 37d2bbfbf6ff00c1daf7e6cb2b17d5d6
I20260811 18:58:09.108304 11161 ts_manager.cc:194] Registered new tserver with Master: c2d72e507d484000b6768f9a7f376ffb (127.29.112.129:37641)
I20260811 18:58:09.109742 11161 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.112.129@KRBTEST.COM'} at 127.29.112.129:41879
WARNING: no policy specified for oryx/127.29.112.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.112.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.112.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/krb5kdc/oryx_127.29.112.130.keytab.
Entry for principal oryx/127.29.112.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/krb5kdc/oryx_127.29.112.130.keytab.
I20260811 18:58:09.166383 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.130:0
--local_ip_for_outbound_sockets=127.29.112.130
--webserver_interface=127.29.112.130
--webserver_port=0
--tserver_master_addrs=127.29.112.190:41859
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/krb5kdc/oryx_127.29.112.130.keytab
--principal=oryx/127.29.112.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:36951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:58:09.522837 11510 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:58:09.523108 11510 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:58:09.523209 11510 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:58:09.530396 11510 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:58:09.530514 11510 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:58:09.530679 11510 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 18:58:09.530768 11510 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.130
I20260811 18:58:09.540423 11510 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:36951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/krb5kdc/oryx_127.29.112.130.keytab
--principal=oryx/127.29.112.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.112.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.112.190:41859
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.130
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:58:09.541633 11510 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:58:09.543279 11510 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:58:09.556631 11518 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:09.556671 11521 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:09.556636 11523 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:58:09.557833 11510 server_base.cc:1034] running on GCE node
I20260811 18:58:09.558498 11510 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:58:09.559450 11510 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:58:09.560712 11510 hybrid_clock.cc:648] HybridClock initialized: now 1786474689560656 us; error 31 us; skew 500 ppm
I20260811 18:58:09.561021 11510 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:58:09 dist-test-slave-h6n0 krb5kdc[11077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474689, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.112.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:58:09.564255 11510 init.cc:377] Logged in from keytab as oryx/127.29.112.130@KRBTEST.COM (short username oryx)
I20260811 18:58:09.565632 11510 webserver.cc:460] Webserver started at http://127.29.112.130:36533/ using document root <none> and password file <none>
I20260811 18:58:09.566313 11510 fs_manager.cc:359] Metadata directory not provided
I20260811 18:58:09.566453 11510 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:58:09.566887 11510 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:58:09.569738 11510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/minicluster-data/ts-1/data/instance:
uuid: "7b050c9777734174b5b4fbc473fef18d"
format_stamp: "Formatted at 2026-08-11 18:58:09 on dist-test-slave-h6n0"
server_key: "eb645c25ca5789551c977187a41d323f"
server_key_iv: "a8691734e023c799e1fda55e8e21f95f"
server_key_version: "encryptionkey@0"
I20260811 18:58:09.570598 11510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/minicluster-data/ts-1/wal/instance:
uuid: "7b050c9777734174b5b4fbc473fef18d"
format_stamp: "Formatted at 2026-08-11 18:58:09 on dist-test-slave-h6n0"
server_key: "eb645c25ca5789551c977187a41d323f"
server_key_iv: "a8691734e023c799e1fda55e8e21f95f"
server_key_version: "encryptionkey@0"
I20260811 18:58:09.575747 11510 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 18:58:09.579718 11532 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:09.581221 11510 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 18:58:09.581431 11510 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/minicluster-data/ts-1/wal
uuid: "7b050c9777734174b5b4fbc473fef18d"
format_stamp: "Formatted at 2026-08-11 18:58:09 on dist-test-slave-h6n0"
server_key: "eb645c25ca5789551c977187a41d323f"
server_key_iv: "a8691734e023c799e1fda55e8e21f95f"
server_key_version: "encryptionkey@0"
I20260811 18:58:09.581671 11510 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:58:09.595844 11510 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:58:09.601100 11510 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:58:09.601619 11510 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:58:09.605263 11510 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:58:09.605410 11510 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:09.605590 11510 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:58:09.605705 11510 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 18:58:09 dist-test-slave-h6n0 krb5kdc[11077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474689, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.112.130@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM
I20260811 18:58:09.633373 11548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:09.606228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.130:38563 (local address 127.29.112.190:41859)
0811 18:58:09.606820 (+ 592us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:09.606830 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:09.608419 (+ 1589us) server_negotiation.cc:407] Connection header received
0811 18:58:09.612072 (+ 3653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:09.612083 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:09.612182 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:09.612270 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:09.615509 (+ 3239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:09.616875 (+ 1366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:09.620206 (+ 3331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:09.620822 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:09.620965 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:09.625601 (+ 4636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:58:09.625609 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:58:09.625616 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:58:09.625666 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:58:09.629169 (+ 3503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:09.630252 (+ 1083us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:09.630258 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:09.630263 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:09.630449 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:09.631048 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:09.631056 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:09.631060 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:09.631332 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:58:09.631516 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:09.632246 (+ 730us) server_negotiation.cc:299] Negotiation successful
0811 18:58:09.632842 (+ 596us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":446,"thread_start_us":314,"threads_started":1}
I20260811 18:58:09.635174 11549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:09.606903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:41859 (local address 127.29.112.130:38563)
0811 18:58:09.608144 (+ 1241us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:09.608247 (+ 103us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:09.611623 (+ 3376us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:09.612939 (+ 1316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:09.612953 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:09.614114 (+ 1161us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:58:09.615228 (+ 1114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:09.615261 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:09.617080 (+ 1819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:09.617086 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:09.620035 (+ 2949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:09.620061 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:09.621094 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:09.621103 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:09.621320 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:09.622043 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:58:09.622065 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:58:09.625301 (+ 3236us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:58:09.629335 (+ 4034us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:58:09.629346 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:58:09.629371 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:58:09.630090 (+ 719us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:58:09.630619 (+ 529us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:58:09.630629 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:58:09.630632 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:58:09.630909 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:58:09.631467 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:58:09.631477 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:58:09.631876 (+ 399us) client_negotiation.cc:769] Sending connection context
0811 18:58:09.632150 (+ 274us) client_negotiation.cc:240] Negotiation successful
0811 18:58:09.633235 (+ 1085us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":731,"thread_start_us":463,"threads_started":1}
I20260811 18:58:09.658234 11548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:09.643561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.130:38039 (local address 127.29.112.190:41859)
0811 18:58:09.643807 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:09.643816 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:09.644369 (+ 553us) server_negotiation.cc:407] Connection header received
0811 18:58:09.644501 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:09.644510 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:09.644614 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:09.644750 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:09.645774 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:09.647215 (+ 1441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:09.649319 (+ 2104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:09.649774 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:09.649916 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:09.652248 (+ 2332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:58:09.652254 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:58:09.652257 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:58:09.652286 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:58:09.655173 (+ 2887us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:09.656203 (+ 1030us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:09.656211 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:09.656215 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:09.656338 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:09.656723 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:09.656728 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:09.656730 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:09.657080 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:58:09.657269 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:09.657647 (+ 378us) server_negotiation.cc:299] Negotiation successful
0811 18:58:09.657868 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 18:58:09.658351 11658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:09.643578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:41859 (local address 127.29.112.130:38039)
0811 18:58:09.644265 (+ 687us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:09.644283 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:09.644400 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:09.644778 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:09.644782 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:09.645281 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:58:09.645628 (+ 347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:09.645639 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:09.647397 (+ 1758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:09.647405 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:09.649140 (+ 1735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:09.649156 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:09.649985 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:09.649991 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:09.650155 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:09.650828 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:58:09.650851 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:58:09.652043 (+ 1192us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:58:09.655381 (+ 3338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:58:09.655387 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:58:09.655393 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:58:09.656032 (+ 639us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:58:09.656465 (+ 433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:58:09.656469 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:58:09.656473 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:58:09.656607 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:58:09.657206 (+ 599us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:58:09.657211 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:58:09.657404 (+ 193us) client_negotiation.cc:769] Sending connection context
0811 18:58:09.657554 (+ 150us) client_negotiation.cc:240] Negotiation successful
0811 18:58:09.657889 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":369,"threads_started":1}
I20260811 18:58:09.669307 11510 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.130:44157
I20260811 18:58:09.669330 11729 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.130:44157 every 8 connection(s)
I20260811 18:58:09.670619 11510 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/minicluster-data/ts-1/data/info.pb
I20260811 18:58:09.672920 11733 heartbeater.cc:347] Connected to a master server at 127.29.112.190:41859
I20260811 18:58:09.673166 11733 heartbeater.cc:464] Registering TS with master...
I20260811 18:58:09.674013 11733 heartbeater.cc:511] Master 127.29.112.190:41859 requested a full tablet report, sending...
I20260811 18:58:09.674141 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 11510
I20260811 18:58:09.674405 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/minicluster-data/ts-1/wal/instance
I20260811 18:58:09.674974 30146 external_mini_cluster.cc:1428] Setting key c14e760fe07da37f36bd5bad8e371815
I20260811 18:58:09.675662 11161 ts_manager.cc:194] Registered new tserver with Master: 7b050c9777734174b5b4fbc473fef18d (127.29.112.130:44157)
I20260811 18:58:09.676487 11161 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.112.130@KRBTEST.COM'} at 127.29.112.130:38563
WARNING: no policy specified for oryx/127.29.112.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.112.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.112.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/krb5kdc/oryx_127.29.112.131.keytab.
Entry for principal oryx/127.29.112.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/krb5kdc/oryx_127.29.112.131.keytab.
I20260811 18:58:09.729889 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.131:0
--local_ip_for_outbound_sockets=127.29.112.131
--webserver_interface=127.29.112.131
--webserver_port=0
--tserver_master_addrs=127.29.112.190:41859
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/krb5kdc/oryx_127.29.112.131.keytab
--principal=oryx/127.29.112.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:36951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:58:10.081238 11738 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:58:10.081528 11738 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:58:10.081640 11738 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:58:10.091022 11738 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:58:10.091174 11738 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:58:10.091259 11738 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 18:58:10.091346 11738 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.131
I20260811 18:58:10.102088 11738 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:36951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/krb5kdc/oryx_127.29.112.131.keytab
--principal=oryx/127.29.112.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.112.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.112.190:41859
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.131
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:58:10.103319 11738 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:58:10.104593 11738 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 18:58:10.112437 11503 heartbeater.cc:503] Master 127.29.112.190:41859 was elected leader, sending a full tablet report...
W20260811 18:58:10.117901 11746 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:10.117972 11755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:10.117977 11750 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:58:10.118834 11738 server_base.cc:1034] running on GCE node
I20260811 18:58:10.119424 11738 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:58:10.120286 11738 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:58:10.121503 11738 hybrid_clock.cc:648] HybridClock initialized: now 1786474690121454 us; error 30 us; skew 500 ppm
I20260811 18:58:10.121752 11738 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:58:10 dist-test-slave-h6n0 krb5kdc[11077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474690, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.112.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:58:10.124934 11738 init.cc:377] Logged in from keytab as oryx/127.29.112.131@KRBTEST.COM (short username oryx)
I20260811 18:58:10.126356 11738 webserver.cc:460] Webserver started at http://127.29.112.131:39847/ using document root <none> and password file <none>
I20260811 18:58:10.127116 11738 fs_manager.cc:359] Metadata directory not provided
I20260811 18:58:10.127207 11738 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:58:10.127477 11738 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:58:10.130214 11738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/minicluster-data/ts-2/data/instance:
uuid: "2134f25fb0da4a129e908b6769b69f76"
format_stamp: "Formatted at 2026-08-11 18:58:10 on dist-test-slave-h6n0"
server_key: "b43840c0bc7b2bd2994f20c1cd5f28af"
server_key_iv: "2bae1425186b5e0461f53b6897803594"
server_key_version: "encryptionkey@0"
I20260811 18:58:10.131047 11738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/minicluster-data/ts-2/wal/instance:
uuid: "2134f25fb0da4a129e908b6769b69f76"
format_stamp: "Formatted at 2026-08-11 18:58:10 on dist-test-slave-h6n0"
server_key: "b43840c0bc7b2bd2994f20c1cd5f28af"
server_key_iv: "2bae1425186b5e0461f53b6897803594"
server_key_version: "encryptionkey@0"
I20260811 18:58:10.135861 11738 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260811 18:58:10.139451 11768 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:10.140695 11738 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 18:58:10.140856 11738 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/minicluster-data/ts-2/wal
uuid: "2134f25fb0da4a129e908b6769b69f76"
format_stamp: "Formatted at 2026-08-11 18:58:10 on dist-test-slave-h6n0"
server_key: "b43840c0bc7b2bd2994f20c1cd5f28af"
server_key_iv: "2bae1425186b5e0461f53b6897803594"
server_key_version: "encryptionkey@0"
I20260811 18:58:10.141086 11738 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:58:10.177048 11738 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:58:10.181452 11738 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:58:10.181687 11738 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:58:10.184492 11738 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:58:10.184609 11738 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:10.184757 11738 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:58:10.184893 11738 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
Aug 11 18:58:10 dist-test-slave-h6n0 krb5kdc[11077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474690, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.112.131@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM
I20260811 18:58:10.215566 11780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:10.185044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.131:49507 (local address 127.29.112.190:41859)
0811 18:58:10.188282 (+ 3238us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:10.188290 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:10.188318 (+ 28us) server_negotiation.cc:407] Connection header received
0811 18:58:10.190497 (+ 2179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:10.190508 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:10.190654 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:10.190760 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:10.193549 (+ 2789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:10.195238 (+ 1689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:10.198733 (+ 3495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:10.199605 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:10.199783 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:10.206044 (+ 6261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:58:10.206054 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:58:10.206062 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:58:10.206122 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:58:10.210658 (+ 4536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:10.212053 (+ 1395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:10.212071 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:10.212077 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:10.212273 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:10.213036 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:10.213040 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:10.213042 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:10.213327 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:58:10.213438 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:10.215000 (+ 1562us) server_negotiation.cc:299] Negotiation successful
0811 18:58:10.215262 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3099,"thread_start_us":3007,"threads_started":1}
I20260811 18:58:10.217218 11783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:10.185719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:41859 (local address 127.29.112.131:49507)
0811 18:58:10.186972 (+ 1253us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:10.187094 (+ 122us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:10.190047 (+ 2953us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:10.191448 (+ 1401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:10.191463 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:10.192363 (+ 900us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:58:10.193247 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:10.193265 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:10.195503 (+ 2238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:10.195514 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:10.198421 (+ 2907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:10.198435 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:10.199893 (+ 1458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:10.199906 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:10.200199 (+ 293us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:10.201267 (+ 1068us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:58:10.201312 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:58:10.205656 (+ 4344us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:58:10.210879 (+ 5223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:58:10.210897 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:58:10.210921 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:58:10.211786 (+ 865us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:58:10.212488 (+ 702us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:58:10.212495 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:58:10.212499 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:58:10.212855 (+ 356us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:58:10.213499 (+ 644us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:58:10.213511 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:58:10.214108 (+ 597us) client_negotiation.cc:769] Sending connection context
0811 18:58:10.214495 (+ 387us) client_negotiation.cc:240] Negotiation successful
0811 18:58:10.215265 (+ 770us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":671,"thread_start_us":415,"threads_started":1}
I20260811 18:58:10.238360 11738 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.131:37327
I20260811 18:58:10.238502 11963 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.131:37327 every 8 connection(s)
I20260811 18:58:10.239817 11738 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/minicluster-data/ts-2/data/info.pb
I20260811 18:58:10.242108 11964 heartbeater.cc:347] Connected to a master server at 127.29.112.190:41859
I20260811 18:58:10.242385 11964 heartbeater.cc:464] Registering TS with master...
I20260811 18:58:10.243099 11964 heartbeater.cc:511] Master 127.29.112.190:41859 requested a full tablet report, sending...
I20260811 18:58:10.244532 11161 ts_manager.cc:194] Registered new tserver with Master: 2134f25fb0da4a129e908b6769b69f76 (127.29.112.131:37327)
I20260811 18:58:10.244973 11780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:10.224699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.131:47171 (local address 127.29.112.190:41859)
0811 18:58:10.225079 (+ 380us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:10.225091 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:10.225690 (+ 599us) server_negotiation.cc:407] Connection header received
0811 18:58:10.225893 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:10.225903 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:10.226047 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:10.226247 (+ 200us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:10.227605 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:10.229074 (+ 1469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:10.232653 (+ 3579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:10.233530 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:10.233696 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:10.237079 (+ 3383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:58:10.237089 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:58:10.237097 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:58:10.237143 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:58:10.241867 (+ 4724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:10.242837 (+ 970us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:10.242843 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:10.242847 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:10.242992 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:10.243401 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:10.243408 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:10.243416 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:10.243700 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:58:10.243876 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:10.244302 (+ 426us) server_negotiation.cc:299] Negotiation successful
0811 18:58:10.244579 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260811 18:58:10.245002 11928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:10.224869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:41859 (local address 127.29.112.131:47171)
0811 18:58:10.225576 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:10.225593 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:10.225712 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:10.226287 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:10.226291 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:10.227058 (+ 767us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:58:10.227417 (+ 359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:10.227427 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:10.229397 (+ 1970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:10.229407 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:10.232354 (+ 2947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:10.232377 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:10.234150 (+ 1773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:10.234161 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:10.234431 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:10.235517 (+ 1086us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:58:10.235548 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:58:10.236764 (+ 1216us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:58:10.242009 (+ 5245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:58:10.242017 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:58:10.242021 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:58:10.242704 (+ 683us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:58:10.243115 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:58:10.243120 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:58:10.243124 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:58:10.243282 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:58:10.243860 (+ 578us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:58:10.243865 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:58:10.244048 (+ 183us) client_negotiation.cc:769] Sending connection context
0811 18:58:10.244189 (+ 141us) client_negotiation.cc:240] Negotiation successful
0811 18:58:10.244526 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":548,"thread_start_us":376,"threads_started":1}
I20260811 18:58:10.245632 11161 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.112.131@KRBTEST.COM'} at 127.29.112.131:49507
I20260811 18:58:10.249346 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 11738
I20260811 18:58:10.249642 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/minicluster-data/ts-2/wal/instance
I20260811 18:58:10.250283 30146 external_mini_cluster.cc:1428] Setting key 9e126aea965101f8b3650aebe7750285
I20260811 18:58:10.253562 30146 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 18:58:10 dist-test-slave-h6n0 krb5kdc[11077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474690, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 18:58:10 dist-test-slave-h6n0 krb5kdc[11077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:58:10 dist-test-slave-h6n0 krb5kdc[11077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
W20260811 18:58:10.321489 11780 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:58:10.308718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52926 (local address 127.29.112.190:41859)
0811 18:58:10.308972 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:10.308979 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:10.309133 (+ 154us) server_negotiation.cc:407] Connection header received
0811 18:58:10.309410 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:10.309415 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:10.309508 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:10.309645 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:10.311323 (+ 1678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:10.312122 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:10.313457 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:10.313909 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:10.314058 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:10.321212 (+ 7154us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52926: BlockingRecv error: recv got EOF from 127.0.0.1:52926 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
Aug 11 18:58:10 dist-test-slave-h6n0 krb5kdc[11077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474690, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM
I20260811 18:58:10.340128 11780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:10.325199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52936 (local address 127.29.112.190:41859)
0811 18:58:10.325460 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:10.325466 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:10.325652 (+ 186us) server_negotiation.cc:407] Connection header received
0811 18:58:10.325797 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:10.325801 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:10.325880 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:10.325970 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:10.327181 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:10.327907 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:10.329140 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:10.329572 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:10.329675 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:10.333534 (+ 3859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:58:10.333539 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:58:10.333543 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:58:10.333577 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:58:10.336510 (+ 2933us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:10.337682 (+ 1172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:10.337687 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:10.337689 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:10.337835 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:10.338302 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:10.338305 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:10.338308 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:10.338695 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:58:10.338923 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:10.339496 (+ 573us) server_negotiation.cc:299] Negotiation successful
0811 18:58:10.339900 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260811 18:58:10.344131 11160 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52936:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:58:10.351507 11160 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:58:10.388595 11991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:10.365336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.131:37327 (local address 127.0.0.1:50966)
0811 18:58:10.378052 (+ 12716us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:10.378121 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:10.378356 (+ 235us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:10.379907 (+ 1551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:10.379915 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:10.379946 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:58:10.380458 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:10.380485 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:10.382830 (+ 2345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:10.382836 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:10.385738 (+ 2902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:10.385753 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:10.387558 (+ 1805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:10.387564 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:10.387754 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:10.387803 (+ 49us) client_negotiation.cc:769] Sending connection context
0811 18:58:10.387955 (+ 152us) client_negotiation.cc:240] Negotiation successful
0811 18:58:10.388149 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12353,"mutex_wait_us":64,"thread_start_us":15619,"threads_started":1}
I20260811 18:58:10.388566 11993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:10.366074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50966 (local address 127.29.112.131:37327)
0811 18:58:10.379180 (+ 13106us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:10.379191 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:10.379233 (+ 42us) server_negotiation.cc:407] Connection header received
0811 18:58:10.379339 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:10.379347 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:10.379616 (+ 269us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:10.380692 (+ 1076us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:58:10.381033 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:10.382702 (+ 1669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:10.385908 (+ 3206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:10.387317 (+ 1409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:10.387536 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:10.387914 (+ 378us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:10.387990 (+ 76us) server_negotiation.cc:299] Negotiation successful
0811 18:58:10.388192 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12935,"thread_start_us":12748,"threads_started":1}
I20260811 18:58:10.388823 11996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:10.368128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43512 (local address 127.29.112.130:44157)
0811 18:58:10.379537 (+ 11409us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:10.379546 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:10.380834 (+ 1288us) server_negotiation.cc:407] Connection header received
0811 18:58:10.381157 (+ 323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:10.381166 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:10.381402 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:10.381565 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:58:10.382377 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:10.383526 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:10.386556 (+ 3030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:10.387459 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:10.387634 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:10.387797 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:10.388340 (+ 543us) server_negotiation.cc:299] Negotiation successful
0811 18:58:10.388497 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11270,"thread_start_us":11128,"threads_started":1}
I20260811 18:58:10.389534 11990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:10.367456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.130:44157 (local address 127.0.0.1:43512)
0811 18:58:10.380726 (+ 13270us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:10.380743 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:10.380948 (+ 205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:10.381822 (+ 874us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:10.381827 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:10.381851 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:58:10.382187 (+ 336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:10.382199 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:10.383873 (+ 1674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:10.383879 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:10.386379 (+ 2500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:10.386412 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:10.387943 (+ 1531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:10.387950 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:10.388119 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:10.388147 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 18:58:10.389011 (+ 864us) client_negotiation.cc:240] Negotiation successful
0811 18:58:10.389132 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13148,"mutex_wait_us":51,"thread_start_us":12260,"threads_started":1}
I20260811 18:58:10.389672 11994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:10.366847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45414 (local address 127.29.112.129:37641)
0811 18:58:10.377417 (+ 10570us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:10.377431 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:10.380137 (+ 2706us) server_negotiation.cc:407] Connection header received
0811 18:58:10.380306 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:10.380314 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:10.380516 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:10.381379 (+ 863us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:58:10.381528 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:10.382691 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:10.385757 (+ 3066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:10.387318 (+ 1561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:10.387604 (+ 286us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:10.387755 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:10.389286 (+ 1531us) server_negotiation.cc:299] Negotiation successful
0811 18:58:10.389417 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10343,"thread_start_us":10170,"threads_started":1}
I20260811 18:58:10.389865 11995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:10.365577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:37641 (local address 127.0.0.1:45414)
0811 18:58:10.380025 (+ 14448us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:10.380043 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:10.380171 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:10.380738 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:10.380743 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:10.380767 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:58:10.381234 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:10.381244 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:10.382898 (+ 1654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:10.382902 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:10.385532 (+ 2630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:10.385545 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:10.388938 (+ 3393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:10.388949 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:10.389104 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:10.389133 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 18:58:10.389224 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 18:58:10.389344 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14301,"mutex_wait_us":162,"thread_start_us":11954,"threads_started":1}
I20260811 18:58:10.393443 11853 tablet_service.cc:1467] Processing CreateTablet for tablet fc81e004eb0b4babaf320b84577cbbe8 (DEFAULT_TABLE table=test-table [id=361e9a5715a446839cf8938717006704]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:58:10.395061 11853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc81e004eb0b4babaf320b84577cbbe8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:58:10.394173 11389 tablet_service.cc:1467] Processing CreateTablet for tablet fc81e004eb0b4babaf320b84577cbbe8 (DEFAULT_TABLE table=test-table [id=361e9a5715a446839cf8938717006704]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:58:10.396487 11389 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc81e004eb0b4babaf320b84577cbbe8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:58:10.396646 11614 tablet_service.cc:1467] Processing CreateTablet for tablet fc81e004eb0b4babaf320b84577cbbe8 (DEFAULT_TABLE table=test-table [id=361e9a5715a446839cf8938717006704]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:58:10.398031 11614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc81e004eb0b4babaf320b84577cbbe8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:58:10.407989 11998 tablet_bootstrap.cc:492] T fc81e004eb0b4babaf320b84577cbbe8 P 2134f25fb0da4a129e908b6769b69f76: Bootstrap starting.
I20260811 18:58:10.411217 11998 tablet_bootstrap.cc:654] T fc81e004eb0b4babaf320b84577cbbe8 P 2134f25fb0da4a129e908b6769b69f76: Neither blocks nor log segments found. Creating new log.
I20260811 18:58:10.412278 11998 log.cc:824] T fc81e004eb0b4babaf320b84577cbbe8 P 2134f25fb0da4a129e908b6769b69f76: Log is configured to *not* fsync() on all Append() calls
I20260811 18:58:10.413375 12001 tablet_bootstrap.cc:492] T fc81e004eb0b4babaf320b84577cbbe8 P c2d72e507d484000b6768f9a7f376ffb: Bootstrap starting.
I20260811 18:58:10.414970 12004 tablet_bootstrap.cc:492] T fc81e004eb0b4babaf320b84577cbbe8 P 7b050c9777734174b5b4fbc473fef18d: Bootstrap starting.
I20260811 18:58:10.417232 12001 tablet_bootstrap.cc:654] T fc81e004eb0b4babaf320b84577cbbe8 P c2d72e507d484000b6768f9a7f376ffb: Neither blocks nor log segments found. Creating new log.
I20260811 18:58:10.418181 12004 tablet_bootstrap.cc:654] T fc81e004eb0b4babaf320b84577cbbe8 P 7b050c9777734174b5b4fbc473fef18d: Neither blocks nor log segments found. Creating new log.
I20260811 18:58:10.418551 12001 log.cc:824] T fc81e004eb0b4babaf320b84577cbbe8 P c2d72e507d484000b6768f9a7f376ffb: Log is configured to *not* fsync() on all Append() calls
I20260811 18:58:10.419023 11998 tablet_bootstrap.cc:492] T fc81e004eb0b4babaf320b84577cbbe8 P 2134f25fb0da4a129e908b6769b69f76: No bootstrap required, opened a new log
I20260811 18:58:10.419387 11998 ts_tablet_manager.cc:1397] T fc81e004eb0b4babaf320b84577cbbe8 P 2134f25fb0da4a129e908b6769b69f76: Time spent bootstrapping tablet: real 0.012s user 0.009s sys 0.000s
I20260811 18:58:10.419399 12004 log.cc:824] T fc81e004eb0b4babaf320b84577cbbe8 P 7b050c9777734174b5b4fbc473fef18d: Log is configured to *not* fsync() on all Append() calls
I20260811 18:58:10.421657 12001 tablet_bootstrap.cc:492] T fc81e004eb0b4babaf320b84577cbbe8 P c2d72e507d484000b6768f9a7f376ffb: No bootstrap required, opened a new log
I20260811 18:58:10.421914 12001 ts_tablet_manager.cc:1397] T fc81e004eb0b4babaf320b84577cbbe8 P c2d72e507d484000b6768f9a7f376ffb: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260811 18:58:10.430357 11998 raft_consensus.cc:348] T fc81e004eb0b4babaf320b84577cbbe8 P 2134f25fb0da4a129e908b6769b69f76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2134f25fb0da4a129e908b6769b69f76" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 37327 } } peers { permanent_uuid: "c2d72e507d484000b6768f9a7f376ffb" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 37641 } } peers { permanent_uuid: "7b050c9777734174b5b4fbc473fef18d" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 44157 } }
I20260811 18:58:10.431082 11998 raft_consensus.cc:374] T fc81e004eb0b4babaf320b84577cbbe8 P 2134f25fb0da4a129e908b6769b69f76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:58:10.431233 11998 raft_consensus.cc:729] T fc81e004eb0b4babaf320b84577cbbe8 P 2134f25fb0da4a129e908b6769b69f76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2134f25fb0da4a129e908b6769b69f76, State: Initialized, Role: FOLLOWER
I20260811 18:58:10.432071 11998 consensus_queue.cc:260] T fc81e004eb0b4babaf320b84577cbbe8 P 2134f25fb0da4a129e908b6769b69f76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2134f25fb0da4a129e908b6769b69f76" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 37327 } } peers { permanent_uuid: "c2d72e507d484000b6768f9a7f376ffb" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 37641 } } peers { permanent_uuid: "7b050c9777734174b5b4fbc473fef18d" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 44157 } }
I20260811 18:58:10.432786 12001 raft_consensus.cc:348] T fc81e004eb0b4babaf320b84577cbbe8 P c2d72e507d484000b6768f9a7f376ffb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2134f25fb0da4a129e908b6769b69f76" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 37327 } } peers { permanent_uuid: "c2d72e507d484000b6768f9a7f376ffb" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 37641 } } peers { permanent_uuid: "7b050c9777734174b5b4fbc473fef18d" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 44157 } }
I20260811 18:58:10.434296 11964 heartbeater.cc:503] Master 127.29.112.190:41859 was elected leader, sending a full tablet report...
I20260811 18:58:10.434504 12001 raft_consensus.cc:374] T fc81e004eb0b4babaf320b84577cbbe8 P c2d72e507d484000b6768f9a7f376ffb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:58:10.434643 12001 raft_consensus.cc:729] T fc81e004eb0b4babaf320b84577cbbe8 P c2d72e507d484000b6768f9a7f376ffb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2d72e507d484000b6768f9a7f376ffb, State: Initialized, Role: FOLLOWER
I20260811 18:58:10.435325 12001 consensus_queue.cc:260] T fc81e004eb0b4babaf320b84577cbbe8 P c2d72e507d484000b6768f9a7f376ffb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2134f25fb0da4a129e908b6769b69f76" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 37327 } } peers { permanent_uuid: "c2d72e507d484000b6768f9a7f376ffb" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 37641 } } peers { permanent_uuid: "7b050c9777734174b5b4fbc473fef18d" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 44157 } }
I20260811 18:58:10.436404 12004 tablet_bootstrap.cc:492] T fc81e004eb0b4babaf320b84577cbbe8 P 7b050c9777734174b5b4fbc473fef18d: No bootstrap required, opened a new log
I20260811 18:58:10.436674 12004 ts_tablet_manager.cc:1397] T fc81e004eb0b4babaf320b84577cbbe8 P 7b050c9777734174b5b4fbc473fef18d: Time spent bootstrapping tablet: real 0.022s user 0.013s sys 0.008s
I20260811 18:58:10.442339 11998 ts_tablet_manager.cc:1428] T fc81e004eb0b4babaf320b84577cbbe8 P 2134f25fb0da4a129e908b6769b69f76: Time spent starting tablet: real 0.023s user 0.010s sys 0.005s
I20260811 18:58:10.444458 12001 ts_tablet_manager.cc:1428] T fc81e004eb0b4babaf320b84577cbbe8 P c2d72e507d484000b6768f9a7f376ffb: Time spent starting tablet: real 0.022s user 0.011s sys 0.008s
I20260811 18:58:10.449834 12004 raft_consensus.cc:348] T fc81e004eb0b4babaf320b84577cbbe8 P 7b050c9777734174b5b4fbc473fef18d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2134f25fb0da4a129e908b6769b69f76" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 37327 } } peers { permanent_uuid: "c2d72e507d484000b6768f9a7f376ffb" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 37641 } } peers { permanent_uuid: "7b050c9777734174b5b4fbc473fef18d" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 44157 } }
I20260811 18:58:10.450505 12004 raft_consensus.cc:374] T fc81e004eb0b4babaf320b84577cbbe8 P 7b050c9777734174b5b4fbc473fef18d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:58:10.450649 12004 raft_consensus.cc:729] T fc81e004eb0b4babaf320b84577cbbe8 P 7b050c9777734174b5b4fbc473fef18d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b050c9777734174b5b4fbc473fef18d, State: Initialized, Role: FOLLOWER
I20260811 18:58:10.451318 12004 consensus_queue.cc:260] T fc81e004eb0b4babaf320b84577cbbe8 P 7b050c9777734174b5b4fbc473fef18d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2134f25fb0da4a129e908b6769b69f76" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 37327 } } peers { permanent_uuid: "c2d72e507d484000b6768f9a7f376ffb" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 37641 } } peers { permanent_uuid: "7b050c9777734174b5b4fbc473fef18d" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 44157 } }
I20260811 18:58:10.452427 11733 heartbeater.cc:503] Master 127.29.112.190:41859 was elected leader, sending a full tablet report...
I20260811 18:58:10.453105 12004 ts_tablet_manager.cc:1428] T fc81e004eb0b4babaf320b84577cbbe8 P 7b050c9777734174b5b4fbc473fef18d: Time spent starting tablet: real 0.016s user 0.007s sys 0.006s
W20260811 18:58:10.492249 11965 tablet.cc:2369] T fc81e004eb0b4babaf320b84577cbbe8 P 2134f25fb0da4a129e908b6769b69f76: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:58:10.512636 12011 raft_consensus.cc:482] T fc81e004eb0b4babaf320b84577cbbe8 P c2d72e507d484000b6768f9a7f376ffb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:58:10.513008 12011 raft_consensus.cc:504] T fc81e004eb0b4babaf320b84577cbbe8 P c2d72e507d484000b6768f9a7f376ffb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2134f25fb0da4a129e908b6769b69f76" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 37327 } } peers { permanent_uuid: "c2d72e507d484000b6768f9a7f376ffb" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 37641 } } peers { permanent_uuid: "7b050c9777734174b5b4fbc473fef18d" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 44157 } }
I20260811 18:58:10.514854 12011 leader_election.cc:288] T fc81e004eb0b4babaf320b84577cbbe8 P c2d72e507d484000b6768f9a7f376ffb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2134f25fb0da4a129e908b6769b69f76 (127.29.112.131:37327), 7b050c9777734174b5b4fbc473fef18d (127.29.112.130:44157)
I20260811 18:58:10.522488 11996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:10.516044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.129:58583 (local address 127.29.112.130:44157)
0811 18:58:10.516265 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:10.516272 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:10.516583 (+ 311us) server_negotiation.cc:407] Connection header received
0811 18:58:10.516771 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:10.516779 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:10.516868 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:10.516968 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:58:10.517702 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:10.518558 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:10.520708 (+ 2150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:10.521475 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:10.521686 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:10.521779 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:10.522056 (+ 277us) server_negotiation.cc:299] Negotiation successful
0811 18:58:10.522192 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 18:58:10.522511 11993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:10.515393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.129:38451 (local address 127.29.112.131:37327)
0811 18:58:10.515630 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:10.515636 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:10.516116 (+ 480us) server_negotiation.cc:407] Connection header received
0811 18:58:10.516446 (+ 330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:10.516452 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:10.516546 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:10.516644 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:58:10.517453 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:10.518269 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:10.520630 (+ 2361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:10.521475 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:10.521665 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:10.521754 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:10.522117 (+ 363us) server_negotiation.cc:299] Negotiation successful
0811 18:58:10.522242 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260811 18:58:10.522624 12015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:10.515694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.130:44157 (local address 127.29.112.129:58583)
0811 18:58:10.516461 (+ 767us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:10.516488 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:10.516626 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:10.517005 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:10.517008 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:10.517032 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:58:10.517535 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:10.517550 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:10.518717 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:10.518722 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:10.520464 (+ 1742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:10.520478 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:10.521734 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:10.521742 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:10.521895 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:10.521920 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 18:58:10.521998 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 18:58:10.522147 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":615,"mutex_wait_us":44,"thread_start_us":433,"threads_started":1}
I20260811 18:58:10.522799 12013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:10.515038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.131:37327 (local address 127.29.112.129:38451)
0811 18:58:10.515969 (+ 931us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:10.516006 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:10.516270 (+ 264us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:10.516696 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:10.516702 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:10.516731 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:58:10.517294 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:10.517310 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:10.518394 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:10.518400 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:10.520468 (+ 2068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:10.520481 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:10.521787 (+ 1306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:10.521795 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:10.521930 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:10.521952 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 18:58:10.522069 (+ 117us) client_negotiation.cc:240] Negotiation successful
0811 18:58:10.522193 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":764,"mutex_wait_us":60,"thread_start_us":481,"threads_started":1}
I20260811 18:58:10.523875 11889 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc81e004eb0b4babaf320b84577cbbe8" candidate_uuid: "c2d72e507d484000b6768f9a7f376ffb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2134f25fb0da4a129e908b6769b69f76" is_pre_election: true
I20260811 18:58:10.524112 11660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc81e004eb0b4babaf320b84577cbbe8" candidate_uuid: "c2d72e507d484000b6768f9a7f376ffb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b050c9777734174b5b4fbc473fef18d" is_pre_election: true
I20260811 18:58:10.524514 11889 raft_consensus.cc:2454] T fc81e004eb0b4babaf320b84577cbbe8 P 2134f25fb0da4a129e908b6769b69f76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2d72e507d484000b6768f9a7f376ffb in term 0.
I20260811 18:58:10.524586 11660 raft_consensus.cc:2454] T fc81e004eb0b4babaf320b84577cbbe8 P 7b050c9777734174b5b4fbc473fef18d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2d72e507d484000b6768f9a7f376ffb in term 0.
I20260811 18:58:10.525327 11301 leader_election.cc:302] T fc81e004eb0b4babaf320b84577cbbe8 P c2d72e507d484000b6768f9a7f376ffb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7b050c9777734174b5b4fbc473fef18d, c2d72e507d484000b6768f9a7f376ffb; no voters:
I20260811 18:58:10.525780 12011 raft_consensus.cc:2781] T fc81e004eb0b4babaf320b84577cbbe8 P c2d72e507d484000b6768f9a7f376ffb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:58:10.525952 12011 raft_consensus.cc:482] T fc81e004eb0b4babaf320b84577cbbe8 P c2d72e507d484000b6768f9a7f376ffb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:58:10.526073 12011 raft_consensus.cc:3037] T fc81e004eb0b4babaf320b84577cbbe8 P c2d72e507d484000b6768f9a7f376ffb [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:58:10.528476 12011 raft_consensus.cc:504] T fc81e004eb0b4babaf320b84577cbbe8 P c2d72e507d484000b6768f9a7f376ffb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2134f25fb0da4a129e908b6769b69f76" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 37327 } } peers { permanent_uuid: "c2d72e507d484000b6768f9a7f376ffb" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 37641 } } peers { permanent_uuid: "7b050c9777734174b5b4fbc473fef18d" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 44157 } }
I20260811 18:58:10.529150 12011 leader_election.cc:288] T fc81e004eb0b4babaf320b84577cbbe8 P c2d72e507d484000b6768f9a7f376ffb [CANDIDATE]: Term 1 election: Requested vote from peers 2134f25fb0da4a129e908b6769b69f76 (127.29.112.131:37327), 7b050c9777734174b5b4fbc473fef18d (127.29.112.130:44157)
I20260811 18:58:10.529942 11889 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc81e004eb0b4babaf320b84577cbbe8" candidate_uuid: "c2d72e507d484000b6768f9a7f376ffb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2134f25fb0da4a129e908b6769b69f76"
I20260811 18:58:10.529942 11660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc81e004eb0b4babaf320b84577cbbe8" candidate_uuid: "c2d72e507d484000b6768f9a7f376ffb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b050c9777734174b5b4fbc473fef18d"
I20260811 18:58:10.530323 11889 raft_consensus.cc:3037] T fc81e004eb0b4babaf320b84577cbbe8 P 2134f25fb0da4a129e908b6769b69f76 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:58:10.530332 11660 raft_consensus.cc:3037] T fc81e004eb0b4babaf320b84577cbbe8 P 7b050c9777734174b5b4fbc473fef18d [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:58:10.534116 11660 raft_consensus.cc:2454] T fc81e004eb0b4babaf320b84577cbbe8 P 7b050c9777734174b5b4fbc473fef18d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2d72e507d484000b6768f9a7f376ffb in term 1.
I20260811 18:58:10.534116 11889 raft_consensus.cc:2454] T fc81e004eb0b4babaf320b84577cbbe8 P 2134f25fb0da4a129e908b6769b69f76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2d72e507d484000b6768f9a7f376ffb in term 1.
I20260811 18:58:10.534675 11304 leader_election.cc:302] T fc81e004eb0b4babaf320b84577cbbe8 P c2d72e507d484000b6768f9a7f376ffb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2134f25fb0da4a129e908b6769b69f76, c2d72e507d484000b6768f9a7f376ffb; no voters:
I20260811 18:58:10.535169 12011 raft_consensus.cc:2781] T fc81e004eb0b4babaf320b84577cbbe8 P c2d72e507d484000b6768f9a7f376ffb [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:58:10.535638 12011 raft_consensus.cc:686] T fc81e004eb0b4babaf320b84577cbbe8 P c2d72e507d484000b6768f9a7f376ffb [term 1 LEADER]: Becoming Leader. State: Replica: c2d72e507d484000b6768f9a7f376ffb, State: Running, Role: LEADER
I20260811 18:58:10.536139 12011 consensus_queue.cc:237] T fc81e004eb0b4babaf320b84577cbbe8 P c2d72e507d484000b6768f9a7f376ffb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2134f25fb0da4a129e908b6769b69f76" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 37327 } } peers { permanent_uuid: "c2d72e507d484000b6768f9a7f376ffb" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 37641 } } peers { permanent_uuid: "7b050c9777734174b5b4fbc473fef18d" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 44157 } }
I20260811 18:58:10.541055 11160 catalog_manager.cc:5495] T fc81e004eb0b4babaf320b84577cbbe8 P c2d72e507d484000b6768f9a7f376ffb reported cstate change: term changed from 0 to 1, leader changed from <none> to c2d72e507d484000b6768f9a7f376ffb (127.29.112.129). New cstate: current_term: 1 leader_uuid: "c2d72e507d484000b6768f9a7f376ffb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2134f25fb0da4a129e908b6769b69f76" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 37327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2d72e507d484000b6768f9a7f376ffb" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 37641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7b050c9777734174b5b4fbc473fef18d" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 44157 } health_report { overall_health: UNKNOWN } } }
I20260811 18:58:10.583362 11994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:10.576167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45430 (local address 127.29.112.129:37641)
0811 18:58:10.576366 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:10.576370 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:10.576517 (+ 147us) server_negotiation.cc:407] Connection header received
0811 18:58:10.576916 (+ 399us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:10.576923 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:10.577081 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:10.577431 (+ 350us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:58:10.577739 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:10.578851 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:10.580495 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:10.580953 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:10.581071 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:10.581737 (+ 666us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:58:10.582405 (+ 668us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:58:10.582635 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:10.582926 (+ 291us) server_negotiation.cc:299] Negotiation successful
0811 18:58:10.583033 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 18:58:10.593833 11889 raft_consensus.cc:1261] T fc81e004eb0b4babaf320b84577cbbe8 P 2134f25fb0da4a129e908b6769b69f76 [term 1 FOLLOWER]: Refusing update from remote peer c2d72e507d484000b6768f9a7f376ffb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:58:10.594061 11660 raft_consensus.cc:1261] T fc81e004eb0b4babaf320b84577cbbe8 P 7b050c9777734174b5b4fbc473fef18d [term 1 FOLLOWER]: Refusing update from remote peer c2d72e507d484000b6768f9a7f376ffb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:58:10.595161 12011 consensus_queue.cc:1035] T fc81e004eb0b4babaf320b84577cbbe8 P c2d72e507d484000b6768f9a7f376ffb [LEADER]: Connected to new peer: Peer: permanent_uuid: "2134f25fb0da4a129e908b6769b69f76" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 37327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:58:10.595498 12016 consensus_queue.cc:1035] T fc81e004eb0b4babaf320b84577cbbe8 P c2d72e507d484000b6768f9a7f376ffb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7b050c9777734174b5b4fbc473fef18d" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 44157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 18:58:10.605314 11505 tablet.cc:2369] T fc81e004eb0b4babaf320b84577cbbe8 P c2d72e507d484000b6768f9a7f376ffb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:58:10.621814 11161 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52936:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:58:10.622121 11161 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52936:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:58:10.626214 11161 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 392e71a7a404425489017441bdfbad94: Sending DeleteTablet for 3 replicas of tablet fc81e004eb0b4babaf320b84577cbbe8
I20260811 18:58:10.627609 11389 tablet_service.cc:1514] Processing DeleteTablet for tablet fc81e004eb0b4babaf320b84577cbbe8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:58:10 UTC) from {username='oryx', principal='oryx/127.29.112.190@KRBTEST.COM'} at 127.0.0.1:45414
I20260811 18:58:10.627771 11614 tablet_service.cc:1514] Processing DeleteTablet for tablet fc81e004eb0b4babaf320b84577cbbe8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:58:10 UTC) from {username='oryx', principal='oryx/127.29.112.190@KRBTEST.COM'} at 127.0.0.1:43512
I20260811 18:58:10.627780 11853 tablet_service.cc:1514] Processing DeleteTablet for tablet fc81e004eb0b4babaf320b84577cbbe8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:58:10 UTC) from {username='oryx', principal='oryx/127.29.112.190@KRBTEST.COM'} at 127.0.0.1:50966
I20260811 18:58:10.631604 11161 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52936:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:58:10.633724 11161 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:58:10.639155 12038 tablet_replica.cc:330] T fc81e004eb0b4babaf320b84577cbbe8 P 2134f25fb0da4a129e908b6769b69f76: stopping tablet replica
I20260811 18:58:10.639690 12038 raft_consensus.cc:2229] T fc81e004eb0b4babaf320b84577cbbe8 P 2134f25fb0da4a129e908b6769b69f76 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:58:10.640231 12038 raft_consensus.cc:2258] T fc81e004eb0b4babaf320b84577cbbe8 P 2134f25fb0da4a129e908b6769b69f76 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:58:10.645465 12041 tablet_replica.cc:330] T fc81e004eb0b4babaf320b84577cbbe8 P 7b050c9777734174b5b4fbc473fef18d: stopping tablet replica
I20260811 18:58:10.646231 11388 tablet_service.cc:1467] Processing CreateTablet for tablet 5568d1f237a7482991df22bb1f1f2832 (DEFAULT_TABLE table=test-table [id=68e2e82dc48f4ef981a3d62f58b2dd97]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:58:10.646464 11852 tablet_service.cc:1467] Processing CreateTablet for tablet 5568d1f237a7482991df22bb1f1f2832 (DEFAULT_TABLE table=test-table [id=68e2e82dc48f4ef981a3d62f58b2dd97]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:58:10.646821 12041 raft_consensus.cc:2229] T fc81e004eb0b4babaf320b84577cbbe8 P 7b050c9777734174b5b4fbc473fef18d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:58:10.647028 11852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5568d1f237a7482991df22bb1f1f2832. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:58:10.647030 11388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5568d1f237a7482991df22bb1f1f2832. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:58:10.647387 12041 raft_consensus.cc:2258] T fc81e004eb0b4babaf320b84577cbbe8 P 7b050c9777734174b5b4fbc473fef18d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:58:10.650753 12041 ts_tablet_manager.cc:1905] T fc81e004eb0b4babaf320b84577cbbe8 P 7b050c9777734174b5b4fbc473fef18d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:58:10.654445 12001 tablet_bootstrap.cc:492] T 5568d1f237a7482991df22bb1f1f2832 P c2d72e507d484000b6768f9a7f376ffb: Bootstrap starting.
I20260811 18:58:10.655087 11614 tablet_service.cc:1467] Processing CreateTablet for tablet 5568d1f237a7482991df22bb1f1f2832 (DEFAULT_TABLE table=test-table [id=68e2e82dc48f4ef981a3d62f58b2dd97]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:58:10.655547 11614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5568d1f237a7482991df22bb1f1f2832. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:58:10.656057 12001 tablet_bootstrap.cc:654] T 5568d1f237a7482991df22bb1f1f2832 P c2d72e507d484000b6768f9a7f376ffb: Neither blocks nor log segments found. Creating new log.
I20260811 18:58:10.658818 11998 tablet_bootstrap.cc:492] T 5568d1f237a7482991df22bb1f1f2832 P 2134f25fb0da4a129e908b6769b69f76: Bootstrap starting.
I20260811 18:58:10.659893 12037 tablet_replica.cc:330] T fc81e004eb0b4babaf320b84577cbbe8 P c2d72e507d484000b6768f9a7f376ffb: stopping tablet replica
I20260811 18:58:10.660751 11998 tablet_bootstrap.cc:654] T 5568d1f237a7482991df22bb1f1f2832 P 2134f25fb0da4a129e908b6769b69f76: Neither blocks nor log segments found. Creating new log.
I20260811 18:58:10.661053 12041 ts_tablet_manager.cc:1918] T fc81e004eb0b4babaf320b84577cbbe8 P 7b050c9777734174b5b4fbc473fef18d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:58:10.661245 12041 log.cc:1196] T fc81e004eb0b4babaf320b84577cbbe8 P 7b050c9777734174b5b4fbc473fef18d: Deleting WAL directory at /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/minicluster-data/ts-1/wal/wals/fc81e004eb0b4babaf320b84577cbbe8
I20260811 18:58:10.661413 12037 raft_consensus.cc:2229] T fc81e004eb0b4babaf320b84577cbbe8 P c2d72e507d484000b6768f9a7f376ffb [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:58:10.661801 12038 ts_tablet_manager.cc:1905] T fc81e004eb0b4babaf320b84577cbbe8 P 2134f25fb0da4a129e908b6769b69f76: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:58:10.661864 12041 ts_tablet_manager.cc:1939] T fc81e004eb0b4babaf320b84577cbbe8 P 7b050c9777734174b5b4fbc473fef18d: Deleting consensus metadata
I20260811 18:58:10.662128 12037 raft_consensus.cc:2258] T fc81e004eb0b4babaf320b84577cbbe8 P c2d72e507d484000b6768f9a7f376ffb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:58:10.664209 11136 catalog_manager.cc:4841] TS 7b050c9777734174b5b4fbc473fef18d (127.29.112.130:44157): tablet fc81e004eb0b4babaf320b84577cbbe8 (table test-table [id=361e9a5715a446839cf8938717006704]) successfully deleted
I20260811 18:58:10.665925 12037 ts_tablet_manager.cc:1905] T fc81e004eb0b4babaf320b84577cbbe8 P c2d72e507d484000b6768f9a7f376ffb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:58:10.668123 11998 tablet_bootstrap.cc:492] T 5568d1f237a7482991df22bb1f1f2832 P 2134f25fb0da4a129e908b6769b69f76: No bootstrap required, opened a new log
I20260811 18:58:10.668130 12004 tablet_bootstrap.cc:492] T 5568d1f237a7482991df22bb1f1f2832 P 7b050c9777734174b5b4fbc473fef18d: Bootstrap starting.
I20260811 18:58:10.668357 11998 ts_tablet_manager.cc:1397] T 5568d1f237a7482991df22bb1f1f2832 P 2134f25fb0da4a129e908b6769b69f76: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.002s
I20260811 18:58:10.669605 11998 raft_consensus.cc:348] T 5568d1f237a7482991df22bb1f1f2832 P 2134f25fb0da4a129e908b6769b69f76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2d72e507d484000b6768f9a7f376ffb" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 37641 } } peers { permanent_uuid: "7b050c9777734174b5b4fbc473fef18d" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 44157 } } peers { permanent_uuid: "2134f25fb0da4a129e908b6769b69f76" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 37327 } }
I20260811 18:58:10.669914 11998 raft_consensus.cc:374] T 5568d1f237a7482991df22bb1f1f2832 P 2134f25fb0da4a129e908b6769b69f76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:58:10.670053 11998 raft_consensus.cc:729] T 5568d1f237a7482991df22bb1f1f2832 P 2134f25fb0da4a129e908b6769b69f76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2134f25fb0da4a129e908b6769b69f76, State: Initialized, Role: FOLLOWER
I20260811 18:58:10.670399 11998 consensus_queue.cc:260] T 5568d1f237a7482991df22bb1f1f2832 P 2134f25fb0da4a129e908b6769b69f76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2d72e507d484000b6768f9a7f376ffb" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 37641 } } peers { permanent_uuid: "7b050c9777734174b5b4fbc473fef18d" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 44157 } } peers { permanent_uuid: "2134f25fb0da4a129e908b6769b69f76" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 37327 } }
I20260811 18:58:10.671872 12004 tablet_bootstrap.cc:654] T 5568d1f237a7482991df22bb1f1f2832 P 7b050c9777734174b5b4fbc473fef18d: Neither blocks nor log segments found. Creating new log.
I20260811 18:58:10.674068 11998 ts_tablet_manager.cc:1428] T 5568d1f237a7482991df22bb1f1f2832 P 2134f25fb0da4a129e908b6769b69f76: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260811 18:58:10.677210 12001 tablet_bootstrap.cc:492] T 5568d1f237a7482991df22bb1f1f2832 P c2d72e507d484000b6768f9a7f376ffb: No bootstrap required, opened a new log
I20260811 18:58:10.677392 12001 ts_tablet_manager.cc:1397] T 5568d1f237a7482991df22bb1f1f2832 P c2d72e507d484000b6768f9a7f376ffb: Time spent bootstrapping tablet: real 0.023s user 0.006s sys 0.005s
I20260811 18:58:10.678704 12001 raft_consensus.cc:348] T 5568d1f237a7482991df22bb1f1f2832 P c2d72e507d484000b6768f9a7f376ffb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2d72e507d484000b6768f9a7f376ffb" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 37641 } } peers { permanent_uuid: "7b050c9777734174b5b4fbc473fef18d" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 44157 } } peers { permanent_uuid: "2134f25fb0da4a129e908b6769b69f76" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 37327 } }
I20260811 18:58:10.679032 12001 raft_consensus.cc:374] T 5568d1f237a7482991df22bb1f1f2832 P c2d72e507d484000b6768f9a7f376ffb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:58:10.679107 12004 tablet_bootstrap.cc:492] T 5568d1f237a7482991df22bb1f1f2832 P 7b050c9777734174b5b4fbc473fef18d: No bootstrap required, opened a new log
I20260811 18:58:10.679216 12001 raft_consensus.cc:729] T 5568d1f237a7482991df22bb1f1f2832 P c2d72e507d484000b6768f9a7f376ffb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2d72e507d484000b6768f9a7f376ffb, State: Initialized, Role: FOLLOWER
I20260811 18:58:10.679373 12004 ts_tablet_manager.cc:1397] T 5568d1f237a7482991df22bb1f1f2832 P 7b050c9777734174b5b4fbc473fef18d: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.001s
I20260811 18:58:10.679538 12001 consensus_queue.cc:260] T 5568d1f237a7482991df22bb1f1f2832 P c2d72e507d484000b6768f9a7f376ffb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2d72e507d484000b6768f9a7f376ffb" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 37641 } } peers { permanent_uuid: "7b050c9777734174b5b4fbc473fef18d" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 44157 } } peers { permanent_uuid: "2134f25fb0da4a129e908b6769b69f76" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 37327 } }
I20260811 18:58:10.680199 12001 ts_tablet_manager.cc:1428] T 5568d1f237a7482991df22bb1f1f2832 P c2d72e507d484000b6768f9a7f376ffb: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260811 18:58:10.680608 12038 ts_tablet_manager.cc:1918] T fc81e004eb0b4babaf320b84577cbbe8 P 2134f25fb0da4a129e908b6769b69f76: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:58:10.680846 12038 log.cc:1196] T fc81e004eb0b4babaf320b84577cbbe8 P 2134f25fb0da4a129e908b6769b69f76: Deleting WAL directory at /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/minicluster-data/ts-2/wal/wals/fc81e004eb0b4babaf320b84577cbbe8
I20260811 18:58:10.681479 12038 ts_tablet_manager.cc:1939] T fc81e004eb0b4babaf320b84577cbbe8 P 2134f25fb0da4a129e908b6769b69f76: Deleting consensus metadata
I20260811 18:58:10.681514 12004 raft_consensus.cc:348] T 5568d1f237a7482991df22bb1f1f2832 P 7b050c9777734174b5b4fbc473fef18d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2d72e507d484000b6768f9a7f376ffb" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 37641 } } peers { permanent_uuid: "7b050c9777734174b5b4fbc473fef18d" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 44157 } } peers { permanent_uuid: "2134f25fb0da4a129e908b6769b69f76" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 37327 } }
I20260811 18:58:10.681850 12004 raft_consensus.cc:374] T 5568d1f237a7482991df22bb1f1f2832 P 7b050c9777734174b5b4fbc473fef18d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:58:10.682015 12004 raft_consensus.cc:729] T 5568d1f237a7482991df22bb1f1f2832 P 7b050c9777734174b5b4fbc473fef18d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b050c9777734174b5b4fbc473fef18d, State: Initialized, Role: FOLLOWER
I20260811 18:58:10.682384 12004 consensus_queue.cc:260] T 5568d1f237a7482991df22bb1f1f2832 P 7b050c9777734174b5b4fbc473fef18d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2d72e507d484000b6768f9a7f376ffb" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 37641 } } peers { permanent_uuid: "7b050c9777734174b5b4fbc473fef18d" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 44157 } } peers { permanent_uuid: "2134f25fb0da4a129e908b6769b69f76" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 37327 } }
I20260811 18:58:10.683276 12037 ts_tablet_manager.cc:1918] T fc81e004eb0b4babaf320b84577cbbe8 P c2d72e507d484000b6768f9a7f376ffb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:58:10.683455 12037 log.cc:1196] T fc81e004eb0b4babaf320b84577cbbe8 P c2d72e507d484000b6768f9a7f376ffb: Deleting WAL directory at /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/minicluster-data/ts-0/wal/wals/fc81e004eb0b4babaf320b84577cbbe8
I20260811 18:58:10.683420 11139 catalog_manager.cc:4841] TS 2134f25fb0da4a129e908b6769b69f76 (127.29.112.131:37327): tablet fc81e004eb0b4babaf320b84577cbbe8 (table test-table [id=361e9a5715a446839cf8938717006704]) successfully deleted
I20260811 18:58:10.684787 12037 ts_tablet_manager.cc:1939] T fc81e004eb0b4babaf320b84577cbbe8 P c2d72e507d484000b6768f9a7f376ffb: Deleting consensus metadata
I20260811 18:58:10.685184 12004 ts_tablet_manager.cc:1428] T 5568d1f237a7482991df22bb1f1f2832 P 7b050c9777734174b5b4fbc473fef18d: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260811 18:58:10.686650 11137 catalog_manager.cc:4841] TS c2d72e507d484000b6768f9a7f376ffb (127.29.112.129:37641): tablet fc81e004eb0b4babaf320b84577cbbe8 (table test-table [id=361e9a5715a446839cf8938717006704]) successfully deleted
I20260811 18:58:10.781944 12016 raft_consensus.cc:482] T 5568d1f237a7482991df22bb1f1f2832 P c2d72e507d484000b6768f9a7f376ffb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:58:10.782378 12016 raft_consensus.cc:504] T 5568d1f237a7482991df22bb1f1f2832 P c2d72e507d484000b6768f9a7f376ffb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2d72e507d484000b6768f9a7f376ffb" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 37641 } } peers { permanent_uuid: "7b050c9777734174b5b4fbc473fef18d" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 44157 } } peers { permanent_uuid: "2134f25fb0da4a129e908b6769b69f76" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 37327 } }
I20260811 18:58:10.783380 12016 leader_election.cc:288] T 5568d1f237a7482991df22bb1f1f2832 P c2d72e507d484000b6768f9a7f376ffb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7b050c9777734174b5b4fbc473fef18d (127.29.112.130:44157), 2134f25fb0da4a129e908b6769b69f76 (127.29.112.131:37327)
I20260811 18:58:10.784153 11660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5568d1f237a7482991df22bb1f1f2832" candidate_uuid: "c2d72e507d484000b6768f9a7f376ffb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b050c9777734174b5b4fbc473fef18d" is_pre_election: true
I20260811 18:58:10.784277 11889 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5568d1f237a7482991df22bb1f1f2832" candidate_uuid: "c2d72e507d484000b6768f9a7f376ffb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2134f25fb0da4a129e908b6769b69f76" is_pre_election: true
I20260811 18:58:10.784467 11660 raft_consensus.cc:2454] T 5568d1f237a7482991df22bb1f1f2832 P 7b050c9777734174b5b4fbc473fef18d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2d72e507d484000b6768f9a7f376ffb in term 0.
I20260811 18:58:10.784634 11889 raft_consensus.cc:2454] T 5568d1f237a7482991df22bb1f1f2832 P 2134f25fb0da4a129e908b6769b69f76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2d72e507d484000b6768f9a7f376ffb in term 0.
I20260811 18:58:10.785239 11304 leader_election.cc:302] T 5568d1f237a7482991df22bb1f1f2832 P c2d72e507d484000b6768f9a7f376ffb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2134f25fb0da4a129e908b6769b69f76, c2d72e507d484000b6768f9a7f376ffb; no voters:
I20260811 18:58:10.785892 12016 raft_consensus.cc:2781] T 5568d1f237a7482991df22bb1f1f2832 P c2d72e507d484000b6768f9a7f376ffb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:58:10.786041 12016 raft_consensus.cc:482] T 5568d1f237a7482991df22bb1f1f2832 P c2d72e507d484000b6768f9a7f376ffb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:58:10.786173 12016 raft_consensus.cc:3037] T 5568d1f237a7482991df22bb1f1f2832 P c2d72e507d484000b6768f9a7f376ffb [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:58:10.788938 12016 raft_consensus.cc:504] T 5568d1f237a7482991df22bb1f1f2832 P c2d72e507d484000b6768f9a7f376ffb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2d72e507d484000b6768f9a7f376ffb" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 37641 } } peers { permanent_uuid: "7b050c9777734174b5b4fbc473fef18d" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 44157 } } peers { permanent_uuid: "2134f25fb0da4a129e908b6769b69f76" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 37327 } }
I20260811 18:58:10.789686 12016 leader_election.cc:288] T 5568d1f237a7482991df22bb1f1f2832 P c2d72e507d484000b6768f9a7f376ffb [CANDIDATE]: Term 1 election: Requested vote from peers 7b050c9777734174b5b4fbc473fef18d (127.29.112.130:44157), 2134f25fb0da4a129e908b6769b69f76 (127.29.112.131:37327)
I20260811 18:58:10.790448 11889 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5568d1f237a7482991df22bb1f1f2832" candidate_uuid: "c2d72e507d484000b6768f9a7f376ffb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2134f25fb0da4a129e908b6769b69f76"
I20260811 18:58:10.790486 11660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5568d1f237a7482991df22bb1f1f2832" candidate_uuid: "c2d72e507d484000b6768f9a7f376ffb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b050c9777734174b5b4fbc473fef18d"
I20260811 18:58:10.790776 11889 raft_consensus.cc:3037] T 5568d1f237a7482991df22bb1f1f2832 P 2134f25fb0da4a129e908b6769b69f76 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:58:10.790922 11660 raft_consensus.cc:3037] T 5568d1f237a7482991df22bb1f1f2832 P 7b050c9777734174b5b4fbc473fef18d [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:58:10.793182 11660 raft_consensus.cc:2454] T 5568d1f237a7482991df22bb1f1f2832 P 7b050c9777734174b5b4fbc473fef18d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2d72e507d484000b6768f9a7f376ffb in term 1.
I20260811 18:58:10.793754 11301 leader_election.cc:302] T 5568d1f237a7482991df22bb1f1f2832 P c2d72e507d484000b6768f9a7f376ffb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7b050c9777734174b5b4fbc473fef18d, c2d72e507d484000b6768f9a7f376ffb; no voters:
I20260811 18:58:10.793968 11889 raft_consensus.cc:2454] T 5568d1f237a7482991df22bb1f1f2832 P 2134f25fb0da4a129e908b6769b69f76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2d72e507d484000b6768f9a7f376ffb in term 1.
I20260811 18:58:10.794160 12016 raft_consensus.cc:2781] T 5568d1f237a7482991df22bb1f1f2832 P c2d72e507d484000b6768f9a7f376ffb [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:58:10.794377 12016 raft_consensus.cc:686] T 5568d1f237a7482991df22bb1f1f2832 P c2d72e507d484000b6768f9a7f376ffb [term 1 LEADER]: Becoming Leader. State: Replica: c2d72e507d484000b6768f9a7f376ffb, State: Running, Role: LEADER
I20260811 18:58:10.794752 12016 consensus_queue.cc:237] T 5568d1f237a7482991df22bb1f1f2832 P c2d72e507d484000b6768f9a7f376ffb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2d72e507d484000b6768f9a7f376ffb" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 37641 } } peers { permanent_uuid: "7b050c9777734174b5b4fbc473fef18d" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 44157 } } peers { permanent_uuid: "2134f25fb0da4a129e908b6769b69f76" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 37327 } }
I20260811 18:58:10.797787 11161 catalog_manager.cc:5495] T 5568d1f237a7482991df22bb1f1f2832 P c2d72e507d484000b6768f9a7f376ffb reported cstate change: term changed from 0 to 1, leader changed from <none> to c2d72e507d484000b6768f9a7f376ffb (127.29.112.129). New cstate: current_term: 1 leader_uuid: "c2d72e507d484000b6768f9a7f376ffb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2d72e507d484000b6768f9a7f376ffb" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 37641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7b050c9777734174b5b4fbc473fef18d" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 44157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2134f25fb0da4a129e908b6769b69f76" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 37327 } health_report { overall_health: UNKNOWN } } }
Aug 11 18:58:10 dist-test-slave-h6n0 krb5kdc[11077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474688, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.112.190@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM
I20260811 18:58:10.870993 12057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:10.844994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:41859 (local address 127.0.0.1:52938)
0811 18:58:10.846102 (+ 1108us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:10.846136 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:10.846369 (+ 233us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:10.846976 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:10.846980 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:10.847734 (+ 754us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:58:10.847998 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:10.848008 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:10.850016 (+ 2008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:10.850020 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:10.851051 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:10.851064 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:10.852148 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:10.852153 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:10.852395 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:10.860994 (+ 8599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:58:10.861025 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:58:10.863500 (+ 2475us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:58:10.868002 (+ 4502us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:58:10.868015 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:58:10.868021 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:58:10.868496 (+ 475us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:58:10.868938 (+ 442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:58:10.868941 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:58:10.868944 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:58:10.869092 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:58:10.869797 (+ 705us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:58:10.869803 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:58:10.869971 (+ 168us) client_negotiation.cc:769] Sending connection context
0811 18:58:10.870118 (+ 147us) client_negotiation.cc:240] Negotiation successful
0811 18:58:10.870473 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":797,"thread_start_us":442,"threads_started":1}
I20260811 18:58:10.871400 12058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:10.845343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52938 (local address 127.29.112.190:41859)
0811 18:58:10.846497 (+ 1154us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:10.846505 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:10.846562 (+ 57us) server_negotiation.cc:407] Connection header received
0811 18:58:10.846636 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:10.846642 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:10.846731 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:10.846891 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:10.848228 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:10.849777 (+ 1549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:10.851228 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:10.851910 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:10.852066 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:10.863801 (+ 11735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:58:10.863810 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:58:10.863816 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:58:10.863858 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:58:10.867808 (+ 3950us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:10.868637 (+ 829us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:10.868642 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:10.868645 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:10.868802 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:10.869208 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:10.869213 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:10.869217 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:10.869619 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:58:10.869777 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:10.870292 (+ 515us) server_negotiation.cc:299] Negotiation successful
0811 18:58:10.870728 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":102,"server-negotiator.queue_time_us":980,"thread_start_us":683,"threads_started":1}
I20260811 18:58:10.877480 11161 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.29.112.190@KRBTEST.COM'} at 127.0.0.1:52938:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 18:58:10.879000 11161 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:58:10.887050 11389 tablet_service.cc:1467] Processing CreateTablet for tablet 8956b657aa814b028f28105d938c3087 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=81cea7308b2c455e987e107edd75db1b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 18:58:10.887504 11614 tablet_service.cc:1467] Processing CreateTablet for tablet 8956b657aa814b028f28105d938c3087 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=81cea7308b2c455e987e107edd75db1b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 18:58:10.887720 11389 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8956b657aa814b028f28105d938c3087. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:58:10.887655 11853 tablet_service.cc:1467] Processing CreateTablet for tablet 8956b657aa814b028f28105d938c3087 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=81cea7308b2c455e987e107edd75db1b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 18:58:10.888094 11614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8956b657aa814b028f28105d938c3087. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:58:10.888257 11853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8956b657aa814b028f28105d938c3087. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:58:10.895484 11998 tablet_bootstrap.cc:492] T 8956b657aa814b028f28105d938c3087 P 2134f25fb0da4a129e908b6769b69f76: Bootstrap starting.
I20260811 18:58:10.895617 12001 tablet_bootstrap.cc:492] T 8956b657aa814b028f28105d938c3087 P c2d72e507d484000b6768f9a7f376ffb: Bootstrap starting.
I20260811 18:58:10.897002 12001 tablet_bootstrap.cc:654] T 8956b657aa814b028f28105d938c3087 P c2d72e507d484000b6768f9a7f376ffb: Neither blocks nor log segments found. Creating new log.
I20260811 18:58:10.897279 11998 tablet_bootstrap.cc:654] T 8956b657aa814b028f28105d938c3087 P 2134f25fb0da4a129e908b6769b69f76: Neither blocks nor log segments found. Creating new log.
I20260811 18:58:10.898507 12004 tablet_bootstrap.cc:492] T 8956b657aa814b028f28105d938c3087 P 7b050c9777734174b5b4fbc473fef18d: Bootstrap starting.
I20260811 18:58:10.899569 12001 tablet_bootstrap.cc:492] T 8956b657aa814b028f28105d938c3087 P c2d72e507d484000b6768f9a7f376ffb: No bootstrap required, opened a new log
I20260811 18:58:10.899762 12001 ts_tablet_manager.cc:1397] T 8956b657aa814b028f28105d938c3087 P c2d72e507d484000b6768f9a7f376ffb: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s
I20260811 18:58:10.900461 12004 tablet_bootstrap.cc:654] T 8956b657aa814b028f28105d938c3087 P 7b050c9777734174b5b4fbc473fef18d: Neither blocks nor log segments found. Creating new log.
I20260811 18:58:10.900580 12001 raft_consensus.cc:348] T 8956b657aa814b028f28105d938c3087 P c2d72e507d484000b6768f9a7f376ffb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2d72e507d484000b6768f9a7f376ffb" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 37641 } } peers { permanent_uuid: "7b050c9777734174b5b4fbc473fef18d" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 44157 } } peers { permanent_uuid: "2134f25fb0da4a129e908b6769b69f76" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 37327 } }
I20260811 18:58:10.900851 12001 raft_consensus.cc:374] T 8956b657aa814b028f28105d938c3087 P c2d72e507d484000b6768f9a7f376ffb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:58:10.900921 12001 raft_consensus.cc:729] T 8956b657aa814b028f28105d938c3087 P c2d72e507d484000b6768f9a7f376ffb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2d72e507d484000b6768f9a7f376ffb, State: Initialized, Role: FOLLOWER
I20260811 18:58:10.901186 12001 consensus_queue.cc:260] T 8956b657aa814b028f28105d938c3087 P c2d72e507d484000b6768f9a7f376ffb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2d72e507d484000b6768f9a7f376ffb" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 37641 } } peers { permanent_uuid: "7b050c9777734174b5b4fbc473fef18d" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 44157 } } peers { permanent_uuid: "2134f25fb0da4a129e908b6769b69f76" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 37327 } }
I20260811 18:58:10.901746 12016 tablet_replica.cc:439] T 8956b657aa814b028f28105d938c3087 P c2d72e507d484000b6768f9a7f376ffb: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2d72e507d484000b6768f9a7f376ffb" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 37641 } } peers { permanent_uuid: "7b050c9777734174b5b4fbc473fef18d" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 44157 } } peers { permanent_uuid: "2134f25fb0da4a129e908b6769b69f76" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 37327 } } }
I20260811 18:58:10.902107 12016 tablet_replica.cc:442] T 8956b657aa814b028f28105d938c3087 P c2d72e507d484000b6768f9a7f376ffb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:58:10.903605 12001 ts_tablet_manager.cc:1428] T 8956b657aa814b028f28105d938c3087 P c2d72e507d484000b6768f9a7f376ffb: Time spent starting tablet: real 0.004s user 0.002s sys 0.001s
I20260811 18:58:10.906935 12004 tablet_bootstrap.cc:492] T 8956b657aa814b028f28105d938c3087 P 7b050c9777734174b5b4fbc473fef18d: No bootstrap required, opened a new log
I20260811 18:58:10.907133 12004 ts_tablet_manager.cc:1397] T 8956b657aa814b028f28105d938c3087 P 7b050c9777734174b5b4fbc473fef18d: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.004s
I20260811 18:58:10.907732 12004 raft_consensus.cc:348] T 8956b657aa814b028f28105d938c3087 P 7b050c9777734174b5b4fbc473fef18d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2d72e507d484000b6768f9a7f376ffb" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 37641 } } peers { permanent_uuid: "7b050c9777734174b5b4fbc473fef18d" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 44157 } } peers { permanent_uuid: "2134f25fb0da4a129e908b6769b69f76" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 37327 } }
I20260811 18:58:10.907951 12004 raft_consensus.cc:374] T 8956b657aa814b028f28105d938c3087 P 7b050c9777734174b5b4fbc473fef18d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:58:10.908090 12004 raft_consensus.cc:729] T 8956b657aa814b028f28105d938c3087 P 7b050c9777734174b5b4fbc473fef18d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b050c9777734174b5b4fbc473fef18d, State: Initialized, Role: FOLLOWER
I20260811 18:58:10.908371 12004 consensus_queue.cc:260] T 8956b657aa814b028f28105d938c3087 P 7b050c9777734174b5b4fbc473fef18d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2d72e507d484000b6768f9a7f376ffb" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 37641 } } peers { permanent_uuid: "7b050c9777734174b5b4fbc473fef18d" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 44157 } } peers { permanent_uuid: "2134f25fb0da4a129e908b6769b69f76" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 37327 } }
I20260811 18:58:10.908844 12012 tablet_replica.cc:439] T 8956b657aa814b028f28105d938c3087 P 7b050c9777734174b5b4fbc473fef18d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2d72e507d484000b6768f9a7f376ffb" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 37641 } } peers { permanent_uuid: "7b050c9777734174b5b4fbc473fef18d" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 44157 } } peers { permanent_uuid: "2134f25fb0da4a129e908b6769b69f76" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 37327 } } }
I20260811 18:58:10.909219 12012 tablet_replica.cc:442] T 8956b657aa814b028f28105d938c3087 P 7b050c9777734174b5b4fbc473fef18d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:58:10.910655 12004 ts_tablet_manager.cc:1428] T 8956b657aa814b028f28105d938c3087 P 7b050c9777734174b5b4fbc473fef18d: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260811 18:58:10.912225 11998 tablet_bootstrap.cc:492] T 8956b657aa814b028f28105d938c3087 P 2134f25fb0da4a129e908b6769b69f76: No bootstrap required, opened a new log
I20260811 18:58:10.912369 11998 ts_tablet_manager.cc:1397] T 8956b657aa814b028f28105d938c3087 P 2134f25fb0da4a129e908b6769b69f76: Time spent bootstrapping tablet: real 0.017s user 0.003s sys 0.011s
I20260811 18:58:10.913539 11998 raft_consensus.cc:348] T 8956b657aa814b028f28105d938c3087 P 2134f25fb0da4a129e908b6769b69f76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2d72e507d484000b6768f9a7f376ffb" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 37641 } } peers { permanent_uuid: "7b050c9777734174b5b4fbc473fef18d" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 44157 } } peers { permanent_uuid: "2134f25fb0da4a129e908b6769b69f76" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 37327 } }
I20260811 18:58:10.913807 11998 raft_consensus.cc:374] T 8956b657aa814b028f28105d938c3087 P 2134f25fb0da4a129e908b6769b69f76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:58:10.913903 11998 raft_consensus.cc:729] T 8956b657aa814b028f28105d938c3087 P 2134f25fb0da4a129e908b6769b69f76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2134f25fb0da4a129e908b6769b69f76, State: Initialized, Role: FOLLOWER
I20260811 18:58:10.914188 11998 consensus_queue.cc:260] T 8956b657aa814b028f28105d938c3087 P 2134f25fb0da4a129e908b6769b69f76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2d72e507d484000b6768f9a7f376ffb" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 37641 } } peers { permanent_uuid: "7b050c9777734174b5b4fbc473fef18d" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 44157 } } peers { permanent_uuid: "2134f25fb0da4a129e908b6769b69f76" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 37327 } }
I20260811 18:58:10.914635 12009 tablet_replica.cc:439] T 8956b657aa814b028f28105d938c3087 P 2134f25fb0da4a129e908b6769b69f76: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2d72e507d484000b6768f9a7f376ffb" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 37641 } } peers { permanent_uuid: "7b050c9777734174b5b4fbc473fef18d" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 44157 } } peers { permanent_uuid: "2134f25fb0da4a129e908b6769b69f76" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 37327 } } }
I20260811 18:58:10.914958 12009 tablet_replica.cc:442] T 8956b657aa814b028f28105d938c3087 P 2134f25fb0da4a129e908b6769b69f76: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:58:10.915323 11998 ts_tablet_manager.cc:1428] T 8956b657aa814b028f28105d938c3087 P 2134f25fb0da4a129e908b6769b69f76: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
W20260811 18:58:10.923641 11734 tablet.cc:2369] T 5568d1f237a7482991df22bb1f1f2832 P 7b050c9777734174b5b4fbc473fef18d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:58:11.222587 12012 raft_consensus.cc:482] T 8956b657aa814b028f28105d938c3087 P 7b050c9777734174b5b4fbc473fef18d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:58:11.222856 12012 raft_consensus.cc:504] T 8956b657aa814b028f28105d938c3087 P 7b050c9777734174b5b4fbc473fef18d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2d72e507d484000b6768f9a7f376ffb" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 37641 } } peers { permanent_uuid: "7b050c9777734174b5b4fbc473fef18d" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 44157 } } peers { permanent_uuid: "2134f25fb0da4a129e908b6769b69f76" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 37327 } }
I20260811 18:58:11.224958 12012 leader_election.cc:288] T 8956b657aa814b028f28105d938c3087 P 7b050c9777734174b5b4fbc473fef18d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2d72e507d484000b6768f9a7f376ffb (127.29.112.129:37641), 2134f25fb0da4a129e908b6769b69f76 (127.29.112.131:37327)
I20260811 18:58:11.236979 12069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:11.225132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.131:37327 (local address 127.29.112.130:45709)
0811 18:58:11.231562 (+ 6430us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:11.231580 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:11.231781 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:11.232159 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:11.232163 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:11.232189 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:58:11.232447 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:11.232456 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:11.233489 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:11.233495 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:11.235058 (+ 1563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:11.235071 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:11.236131 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:11.236137 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:11.236310 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:11.236334 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 18:58:11.236451 (+ 117us) client_negotiation.cc:240] Negotiation successful
0811 18:58:11.236590 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6331,"threads_started":1}
I20260811 18:58:11.236961 12072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:11.225512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.130:45709 (local address 127.29.112.131:37327)
0811 18:58:11.231036 (+ 5524us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:11.231047 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:11.231699 (+ 652us) server_negotiation.cc:407] Connection header received
0811 18:58:11.231941 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:11.231947 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:11.232030 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:11.232138 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:58:11.232605 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:11.233338 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:11.235210 (+ 1872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:11.235970 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:11.236141 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:11.236217 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:11.236477 (+ 260us) server_negotiation.cc:299] Negotiation successful
0811 18:58:11.236614 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5337,"thread_start_us":5182,"threads_started":1}
I20260811 18:58:11.239148 11889 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8956b657aa814b028f28105d938c3087" candidate_uuid: "7b050c9777734174b5b4fbc473fef18d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2134f25fb0da4a129e908b6769b69f76" is_pre_election: true
I20260811 18:58:11.239450 11889 raft_consensus.cc:2454] T 8956b657aa814b028f28105d938c3087 P 2134f25fb0da4a129e908b6769b69f76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7b050c9777734174b5b4fbc473fef18d in term 0.
I20260811 18:58:11.240003 11538 leader_election.cc:302] T 8956b657aa814b028f28105d938c3087 P 7b050c9777734174b5b4fbc473fef18d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2134f25fb0da4a129e908b6769b69f76, 7b050c9777734174b5b4fbc473fef18d; no voters:
I20260811 18:58:11.240463 12012 raft_consensus.cc:2781] T 8956b657aa814b028f28105d938c3087 P 7b050c9777734174b5b4fbc473fef18d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:58:11.240594 12012 raft_consensus.cc:482] T 8956b657aa814b028f28105d938c3087 P 7b050c9777734174b5b4fbc473fef18d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:58:11.240690 12012 raft_consensus.cc:3037] T 8956b657aa814b028f28105d938c3087 P 7b050c9777734174b5b4fbc473fef18d [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:58:11.243448 12012 raft_consensus.cc:504] T 8956b657aa814b028f28105d938c3087 P 7b050c9777734174b5b4fbc473fef18d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2d72e507d484000b6768f9a7f376ffb" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 37641 } } peers { permanent_uuid: "7b050c9777734174b5b4fbc473fef18d" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 44157 } } peers { permanent_uuid: "2134f25fb0da4a129e908b6769b69f76" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 37327 } }
I20260811 18:58:11.243950 12069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:11.225132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:37641 (local address 127.29.112.130:46805)
0811 18:58:11.237462 (+ 12330us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:11.237477 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:11.237629 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:11.238149 (+ 520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:11.238152 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:11.238171 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:58:11.238320 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:11.238328 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:11.239580 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:11.239585 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:11.241449 (+ 1864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:11.241464 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:11.243059 (+ 1595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:11.243065 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:11.243267 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:11.243313 (+ 46us) client_negotiation.cc:769] Sending connection context
0811 18:58:11.243422 (+ 109us) client_negotiation.cc:240] Negotiation successful
0811 18:58:11.243565 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12254,"mutex_wait_us":236,"thread_start_us":5995,"threads_started":1}
I20260811 18:58:11.244117 12071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:11.225389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.130:46805 (local address 127.29.112.129:37641)
0811 18:58:11.231443 (+ 6054us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:11.231449 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:11.237583 (+ 6134us) server_negotiation.cc:407] Connection header received
0811 18:58:11.237771 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:11.237778 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:11.237878 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:11.237977 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:58:11.238461 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:11.239440 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:11.241647 (+ 2207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:11.242886 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:11.243042 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:11.243112 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:11.243451 (+ 339us) server_negotiation.cc:299] Negotiation successful
0811 18:58:11.243601 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5916,"thread_start_us":5790,"threads_started":1}
I20260811 18:58:11.244881 12012 leader_election.cc:288] T 8956b657aa814b028f28105d938c3087 P 7b050c9777734174b5b4fbc473fef18d [CANDIDATE]: Term 1 election: Requested vote from peers c2d72e507d484000b6768f9a7f376ffb (127.29.112.129:37641), 2134f25fb0da4a129e908b6769b69f76 (127.29.112.131:37327)
I20260811 18:58:11.244971 11426 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8956b657aa814b028f28105d938c3087" candidate_uuid: "7b050c9777734174b5b4fbc473fef18d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2d72e507d484000b6768f9a7f376ffb" is_pre_election: true
I20260811 18:58:11.245393 11425 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8956b657aa814b028f28105d938c3087" candidate_uuid: "7b050c9777734174b5b4fbc473fef18d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2d72e507d484000b6768f9a7f376ffb"
I20260811 18:58:11.245558 11426 raft_consensus.cc:2454] T 8956b657aa814b028f28105d938c3087 P c2d72e507d484000b6768f9a7f376ffb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7b050c9777734174b5b4fbc473fef18d in term 0.
I20260811 18:58:11.245604 11889 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8956b657aa814b028f28105d938c3087" candidate_uuid: "7b050c9777734174b5b4fbc473fef18d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2134f25fb0da4a129e908b6769b69f76"
I20260811 18:58:11.245832 11889 raft_consensus.cc:3037] T 8956b657aa814b028f28105d938c3087 P 2134f25fb0da4a129e908b6769b69f76 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:58:11.247979 11889 raft_consensus.cc:2454] T 8956b657aa814b028f28105d938c3087 P 2134f25fb0da4a129e908b6769b69f76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7b050c9777734174b5b4fbc473fef18d in term 1.
I20260811 18:58:11.248377 11538 leader_election.cc:302] T 8956b657aa814b028f28105d938c3087 P 7b050c9777734174b5b4fbc473fef18d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2134f25fb0da4a129e908b6769b69f76, 7b050c9777734174b5b4fbc473fef18d; no voters: c2d72e507d484000b6768f9a7f376ffb
I20260811 18:58:11.248708 12012 raft_consensus.cc:2781] T 8956b657aa814b028f28105d938c3087 P 7b050c9777734174b5b4fbc473fef18d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:58:11.249179 12012 raft_consensus.cc:686] T 8956b657aa814b028f28105d938c3087 P 7b050c9777734174b5b4fbc473fef18d [term 1 LEADER]: Becoming Leader. State: Replica: 7b050c9777734174b5b4fbc473fef18d, State: Running, Role: LEADER
I20260811 18:58:11.249598 12012 consensus_queue.cc:237] T 8956b657aa814b028f28105d938c3087 P 7b050c9777734174b5b4fbc473fef18d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2d72e507d484000b6768f9a7f376ffb" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 37641 } } peers { permanent_uuid: "7b050c9777734174b5b4fbc473fef18d" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 44157 } } peers { permanent_uuid: "2134f25fb0da4a129e908b6769b69f76" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 37327 } }
I20260811 18:58:11.251423 12073 tablet_replica.cc:439] T 8956b657aa814b028f28105d938c3087 P 7b050c9777734174b5b4fbc473fef18d: TxnStatusTablet state changed. Reason: New leader 7b050c9777734174b5b4fbc473fef18d. Latest consensus state: current_term: 1 leader_uuid: "7b050c9777734174b5b4fbc473fef18d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2d72e507d484000b6768f9a7f376ffb" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 37641 } } peers { permanent_uuid: "7b050c9777734174b5b4fbc473fef18d" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 44157 } } peers { permanent_uuid: "2134f25fb0da4a129e908b6769b69f76" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 37327 } } }
I20260811 18:58:11.251801 12073 tablet_replica.cc:442] T 8956b657aa814b028f28105d938c3087 P 7b050c9777734174b5b4fbc473fef18d: This TxnStatusTablet replica's current role is: LEADER
I20260811 18:58:11.253338 11161 catalog_manager.cc:5495] T 8956b657aa814b028f28105d938c3087 P 7b050c9777734174b5b4fbc473fef18d reported cstate change: term changed from 0 to 1, leader changed from <none> to 7b050c9777734174b5b4fbc473fef18d (127.29.112.130). New cstate: current_term: 1 leader_uuid: "7b050c9777734174b5b4fbc473fef18d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2d72e507d484000b6768f9a7f376ffb" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 37641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7b050c9777734174b5b4fbc473fef18d" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 44157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2134f25fb0da4a129e908b6769b69f76" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 37327 } health_report { overall_health: UNKNOWN } } }
I20260811 18:58:11.254976 12076 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 18:58:11.255191 12076 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 18:58:11.297000 12057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:11.290354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.130:44157 (local address 127.0.0.1:43520)
0811 18:58:11.290634 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:11.290658 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:11.290779 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:11.291691 (+ 912us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:11.291696 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:11.291709 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 18:58:11.291963 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:11.291973 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:11.293144 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:11.293150 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:11.294165 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:11.294176 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:11.295085 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:11.295091 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:11.295287 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:11.295353 (+ 66us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 18:58:11.296371 (+ 1018us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 18:58:11.296396 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 18:58:11.296519 (+ 123us) client_negotiation.cc:240] Negotiation successful
0811 18:58:11.296677 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260811 18:58:11.297123 12079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:11.290567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43520 (local address 127.29.112.130:44157)
0811 18:58:11.291301 (+ 734us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:11.291305 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:11.291325 (+ 20us) server_negotiation.cc:407] Connection header received
0811 18:58:11.291409 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:11.291412 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:11.291493 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:11.291665 (+ 172us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:58:11.292123 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:11.292994 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:11.294310 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:11.294874 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:11.295048 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:11.295612 (+ 564us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:58:11.296236 (+ 624us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:58:11.296406 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:11.296566 (+ 160us) server_negotiation.cc:299] Negotiation successful
0811 18:58:11.296652 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":610,"thread_start_us":444,"threads_started":1}
I20260811 18:58:11.302321 11425 raft_consensus.cc:3037] T 8956b657aa814b028f28105d938c3087 P c2d72e507d484000b6768f9a7f376ffb [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:58:11.302596 11889 raft_consensus.cc:1261] T 8956b657aa814b028f28105d938c3087 P 2134f25fb0da4a129e908b6769b69f76 [term 1 FOLLOWER]: Refusing update from remote peer 7b050c9777734174b5b4fbc473fef18d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:58:11.303382 12073 consensus_queue.cc:1035] T 8956b657aa814b028f28105d938c3087 P 7b050c9777734174b5b4fbc473fef18d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2134f25fb0da4a129e908b6769b69f76" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 37327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:58:11.305461 11425 raft_consensus.cc:1261] T 8956b657aa814b028f28105d938c3087 P c2d72e507d484000b6768f9a7f376ffb [term 1 FOLLOWER]: Refusing update from remote peer 7b050c9777734174b5b4fbc473fef18d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:58:11.306290 12073 consensus_queue.cc:1035] T 8956b657aa814b028f28105d938c3087 P 7b050c9777734174b5b4fbc473fef18d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2d72e507d484000b6768f9a7f376ffb" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 37641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:58:11.307245 12009 tablet_replica.cc:439] T 8956b657aa814b028f28105d938c3087 P 2134f25fb0da4a129e908b6769b69f76: TxnStatusTablet state changed. Reason: New leader 7b050c9777734174b5b4fbc473fef18d. Latest consensus state: current_term: 1 leader_uuid: "7b050c9777734174b5b4fbc473fef18d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2d72e507d484000b6768f9a7f376ffb" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 37641 } } peers { permanent_uuid: "7b050c9777734174b5b4fbc473fef18d" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 44157 } } peers { permanent_uuid: "2134f25fb0da4a129e908b6769b69f76" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 37327 } } }
I20260811 18:58:11.307526 12009 tablet_replica.cc:442] T 8956b657aa814b028f28105d938c3087 P 2134f25fb0da4a129e908b6769b69f76: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:58:11.310341 12016 tablet_replica.cc:439] T 8956b657aa814b028f28105d938c3087 P c2d72e507d484000b6768f9a7f376ffb: TxnStatusTablet state changed. Reason: New leader 7b050c9777734174b5b4fbc473fef18d. Latest consensus state: current_term: 1 leader_uuid: "7b050c9777734174b5b4fbc473fef18d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2d72e507d484000b6768f9a7f376ffb" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 37641 } } peers { permanent_uuid: "7b050c9777734174b5b4fbc473fef18d" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 44157 } } peers { permanent_uuid: "2134f25fb0da4a129e908b6769b69f76" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 37327 } } }
I20260811 18:58:11.310652 12016 tablet_replica.cc:442] T 8956b657aa814b028f28105d938c3087 P c2d72e507d484000b6768f9a7f376ffb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:58:11.311204 12081 mvcc.cc:205] Tried to move back new op lower bound from 7317400335564984320 to 7317400335361531904. Current Snapshot: MvccSnapshot[applied={T|T < 7317400335564984320}]
I20260811 18:58:11.313755 12009 tablet_replica.cc:439] T 8956b657aa814b028f28105d938c3087 P 2134f25fb0da4a129e908b6769b69f76: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7b050c9777734174b5b4fbc473fef18d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2d72e507d484000b6768f9a7f376ffb" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 37641 } } peers { permanent_uuid: "7b050c9777734174b5b4fbc473fef18d" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 44157 } } peers { permanent_uuid: "2134f25fb0da4a129e908b6769b69f76" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 37327 } } }
I20260811 18:58:11.314030 12009 tablet_replica.cc:442] T 8956b657aa814b028f28105d938c3087 P 2134f25fb0da4a129e908b6769b69f76: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:58:11.314841 12016 tablet_replica.cc:439] T 8956b657aa814b028f28105d938c3087 P c2d72e507d484000b6768f9a7f376ffb: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7b050c9777734174b5b4fbc473fef18d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2d72e507d484000b6768f9a7f376ffb" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 37641 } } peers { permanent_uuid: "7b050c9777734174b5b4fbc473fef18d" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 44157 } } peers { permanent_uuid: "2134f25fb0da4a129e908b6769b69f76" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 37327 } } }
I20260811 18:58:11.315104 12016 tablet_replica.cc:442] T 8956b657aa814b028f28105d938c3087 P c2d72e507d484000b6768f9a7f376ffb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:58:11.317870 12082 mvcc.cc:205] Tried to move back new op lower bound from 7317400335564984320 to 7317400335361531904. Current Snapshot: MvccSnapshot[applied={T|T < 7317400335564984320 or (T in {7317400335564984320})}]
I20260811 18:58:11.320062 12097 tablet_replica.cc:439] T 8956b657aa814b028f28105d938c3087 P 7b050c9777734174b5b4fbc473fef18d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7b050c9777734174b5b4fbc473fef18d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2d72e507d484000b6768f9a7f376ffb" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 37641 } } peers { permanent_uuid: "7b050c9777734174b5b4fbc473fef18d" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 44157 } } peers { permanent_uuid: "2134f25fb0da4a129e908b6769b69f76" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 37327 } } }
I20260811 18:58:11.320608 12097 tablet_replica.cc:442] T 8956b657aa814b028f28105d938c3087 P 7b050c9777734174b5b4fbc473fef18d: This TxnStatusTablet replica's current role is: LEADER
I20260811 18:58:11.320935 12012 tablet_replica.cc:439] T 8956b657aa814b028f28105d938c3087 P 7b050c9777734174b5b4fbc473fef18d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7b050c9777734174b5b4fbc473fef18d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2d72e507d484000b6768f9a7f376ffb" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 37641 } } peers { permanent_uuid: "7b050c9777734174b5b4fbc473fef18d" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 44157 } } peers { permanent_uuid: "2134f25fb0da4a129e908b6769b69f76" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 37327 } } }
I20260811 18:58:11.321218 12012 tablet_replica.cc:442] T 8956b657aa814b028f28105d938c3087 P 7b050c9777734174b5b4fbc473fef18d: This TxnStatusTablet replica's current role is: LEADER
I20260811 18:58:11.331198 12016 consensus_queue.cc:1035] T 5568d1f237a7482991df22bb1f1f2832 P c2d72e507d484000b6768f9a7f376ffb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7b050c9777734174b5b4fbc473fef18d" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 44157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:58:11.342746 12103 consensus_queue.cc:1035] T 5568d1f237a7482991df22bb1f1f2832 P c2d72e507d484000b6768f9a7f376ffb [LEADER]: Connected to new peer: Peer: permanent_uuid: "2134f25fb0da4a129e908b6769b69f76" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 37327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:58:11.356397 12079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:11.349528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.129:38571 (local address 127.29.112.130:44157)
0811 18:58:11.349724 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:11.349732 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:11.350200 (+ 468us) server_negotiation.cc:407] Connection header received
0811 18:58:11.350479 (+ 279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:11.350482 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:11.350631 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:11.350715 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:58:11.351430 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:11.352258 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:11.354639 (+ 2381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:11.355094 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:11.355201 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:11.355767 (+ 566us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:58:11.355892 (+ 125us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:58:11.355977 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:11.356116 (+ 139us) server_negotiation.cc:299] Negotiation successful
0811 18:58:11.356178 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 18:58:11.356648 12108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:11.349042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.130:44157 (local address 127.29.112.129:38571)
0811 18:58:11.350023 (+ 981us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:11.350061 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:11.350336 (+ 275us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:11.350781 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:11.350788 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:11.350806 (+ 18us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 18:58:11.351213 (+ 407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:11.351228 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:11.352424 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:11.352430 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:11.354388 (+ 1958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:11.354410 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:11.355285 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:11.355292 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:11.355473 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:11.355557 (+ 84us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 18:58:11.355983 (+ 426us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 18:58:11.355998 (+ 15us) client_negotiation.cc:769] Sending connection context
0811 18:58:11.356086 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 18:58:11.356239 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":597,"thread_start_us":702,"threads_started":1}
I20260811 18:58:11.403003 12071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:11.395880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.130:45157 (local address 127.29.112.129:37641)
0811 18:58:11.396099 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:11.396105 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:11.396749 (+ 644us) server_negotiation.cc:407] Connection header received
0811 18:58:11.396909 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:11.396915 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:11.397014 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:11.397110 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:58:11.397633 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:11.398783 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:11.400778 (+ 1995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:11.401434 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:11.401566 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:11.402153 (+ 587us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:58:11.402286 (+ 133us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:58:11.402379 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:11.402594 (+ 215us) server_negotiation.cc:299] Negotiation successful
0811 18:58:11.402685 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 18:58:11.403395 12112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:11.395864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:37641 (local address 127.29.112.130:45157)
0811 18:58:11.396645 (+ 781us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:11.396665 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:11.396782 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:11.397129 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:11.397132 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:11.397141 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 18:58:11.397475 (+ 334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:11.397482 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:11.398891 (+ 1409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:11.398895 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:11.400637 (+ 1742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:11.400651 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:11.401655 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:11.401664 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:11.401826 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:11.401927 (+ 101us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 18:58:11.402408 (+ 481us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 18:58:11.402420 (+ 12us) client_negotiation.cc:769] Sending connection context
0811 18:58:11.402517 (+ 97us) client_negotiation.cc:240] Negotiation successful
0811 18:58:11.402687 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":561,"thread_start_us":393,"threads_started":1}
I20260811 18:58:11.449826 11160 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52936:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:58:11.450112 11160 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52936:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:58:11.453580 11160 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 392e71a7a404425489017441bdfbad94: Sending DeleteTablet for 3 replicas of tablet 5568d1f237a7482991df22bb1f1f2832
I20260811 18:58:11.454407 11389 tablet_service.cc:1514] Processing DeleteTablet for tablet 5568d1f237a7482991df22bb1f1f2832 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:58:11 UTC) from {username='oryx', principal='oryx/127.29.112.190@KRBTEST.COM'} at 127.0.0.1:45414
I20260811 18:58:11.454643 11853 tablet_service.cc:1514] Processing DeleteTablet for tablet 5568d1f237a7482991df22bb1f1f2832 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:58:11 UTC) from {username='oryx', principal='oryx/127.29.112.190@KRBTEST.COM'} at 127.0.0.1:50966
I20260811 18:58:11.455744 11614 tablet_service.cc:1514] Processing DeleteTablet for tablet 5568d1f237a7482991df22bb1f1f2832 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:58:11 UTC) from {username='oryx', principal='oryx/127.29.112.190@KRBTEST.COM'} at 127.0.0.1:43512
I20260811 18:58:11.467034 12117 tablet_replica.cc:330] T 5568d1f237a7482991df22bb1f1f2832 P 7b050c9777734174b5b4fbc473fef18d: stopping tablet replica
I20260811 18:58:11.467032 12116 tablet_replica.cc:330] T 5568d1f237a7482991df22bb1f1f2832 P 2134f25fb0da4a129e908b6769b69f76: stopping tablet replica
I20260811 18:58:11.467288 12117 raft_consensus.cc:2229] T 5568d1f237a7482991df22bb1f1f2832 P 7b050c9777734174b5b4fbc473fef18d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:58:11.467274 12116 raft_consensus.cc:2229] T 5568d1f237a7482991df22bb1f1f2832 P 2134f25fb0da4a129e908b6769b69f76 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:58:11.467584 12117 raft_consensus.cc:2258] T 5568d1f237a7482991df22bb1f1f2832 P 7b050c9777734174b5b4fbc473fef18d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:58:11.467617 12116 raft_consensus.cc:2258] T 5568d1f237a7482991df22bb1f1f2832 P 2134f25fb0da4a129e908b6769b69f76 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:58:11.468863 12116 ts_tablet_manager.cc:1905] T 5568d1f237a7482991df22bb1f1f2832 P 2134f25fb0da4a129e908b6769b69f76: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:58:11.470618 12115 tablet_replica.cc:330] T 5568d1f237a7482991df22bb1f1f2832 P c2d72e507d484000b6768f9a7f376ffb: stopping tablet replica
I20260811 18:58:11.470894 12115 raft_consensus.cc:2229] T 5568d1f237a7482991df22bb1f1f2832 P c2d72e507d484000b6768f9a7f376ffb [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:58:11.471351 12115 raft_consensus.cc:2258] T 5568d1f237a7482991df22bb1f1f2832 P c2d72e507d484000b6768f9a7f376ffb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:58:11.472427 12117 ts_tablet_manager.cc:1905] T 5568d1f237a7482991df22bb1f1f2832 P 7b050c9777734174b5b4fbc473fef18d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:58:11.474223 12115 ts_tablet_manager.cc:1905] T 5568d1f237a7482991df22bb1f1f2832 P c2d72e507d484000b6768f9a7f376ffb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:58:11.479257 12115 ts_tablet_manager.cc:1918] T 5568d1f237a7482991df22bb1f1f2832 P c2d72e507d484000b6768f9a7f376ffb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 18:58:11.479441 12115 log.cc:1196] T 5568d1f237a7482991df22bb1f1f2832 P c2d72e507d484000b6768f9a7f376ffb: Deleting WAL directory at /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/minicluster-data/ts-0/wal/wals/5568d1f237a7482991df22bb1f1f2832
I20260811 18:58:11.479974 12115 ts_tablet_manager.cc:1939] T 5568d1f237a7482991df22bb1f1f2832 P c2d72e507d484000b6768f9a7f376ffb: Deleting consensus metadata
I20260811 18:58:11.480783 12117 ts_tablet_manager.cc:1918] T 5568d1f237a7482991df22bb1f1f2832 P 7b050c9777734174b5b4fbc473fef18d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 18:58:11.480944 12117 log.cc:1196] T 5568d1f237a7482991df22bb1f1f2832 P 7b050c9777734174b5b4fbc473fef18d: Deleting WAL directory at /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/minicluster-data/ts-1/wal/wals/5568d1f237a7482991df22bb1f1f2832
I20260811 18:58:11.481200 11137 catalog_manager.cc:4841] TS c2d72e507d484000b6768f9a7f376ffb (127.29.112.129:37641): tablet 5568d1f237a7482991df22bb1f1f2832 (table test-table [id=68e2e82dc48f4ef981a3d62f58b2dd97]) successfully deleted
I20260811 18:58:11.481426 12117 ts_tablet_manager.cc:1939] T 5568d1f237a7482991df22bb1f1f2832 P 7b050c9777734174b5b4fbc473fef18d: Deleting consensus metadata
I20260811 18:58:11.481788 12116 ts_tablet_manager.cc:1918] T 5568d1f237a7482991df22bb1f1f2832 P 2134f25fb0da4a129e908b6769b69f76: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 18:58:11.481940 12116 log.cc:1196] T 5568d1f237a7482991df22bb1f1f2832 P 2134f25fb0da4a129e908b6769b69f76: Deleting WAL directory at /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/minicluster-data/ts-2/wal/wals/5568d1f237a7482991df22bb1f1f2832
I20260811 18:58:11.482378 12116 ts_tablet_manager.cc:1939] T 5568d1f237a7482991df22bb1f1f2832 P 2134f25fb0da4a129e908b6769b69f76: Deleting consensus metadata
I20260811 18:58:11.482594 11136 catalog_manager.cc:4841] TS 7b050c9777734174b5b4fbc473fef18d (127.29.112.130:44157): tablet 5568d1f237a7482991df22bb1f1f2832 (table test-table [id=68e2e82dc48f4ef981a3d62f58b2dd97]) successfully deleted
I20260811 18:58:11.483351 11139 catalog_manager.cc:4841] TS 2134f25fb0da4a129e908b6769b69f76 (127.29.112.131:37327): tablet 5568d1f237a7482991df22bb1f1f2832 (table test-table [id=68e2e82dc48f4ef981a3d62f58b2dd97]) successfully deleted
Aug 11 18:58:11 dist-test-slave-h6n0 krb5kdc[11077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474691, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 18:58:11 dist-test-slave-h6n0 krb5kdc[11077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:58:11 dist-test-slave-h6n0 krb5kdc[11077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
W20260811 18:58:11.537536 12129 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:58:11.526200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52948 (local address 127.29.112.190:41859)
0811 18:58:11.527258 (+ 1058us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:11.527263 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:11.527280 (+ 17us) server_negotiation.cc:407] Connection header received
0811 18:58:11.527337 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:11.527341 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:11.527403 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:11.527528 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:11.529010 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:11.529764 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:11.531154 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:11.531588 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:11.531730 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:11.537191 (+ 5461us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52948: BlockingRecv error: recv got EOF from 127.0.0.1:52948 (error 108)
Metrics: {"server-negotiator.queue_time_us":898,"thread_start_us":732,"threads_started":1}
Aug 11 18:58:11 dist-test-slave-h6n0 krb5kdc[11077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474691, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM
I20260811 18:58:11.556959 12129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:11.541192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52964 (local address 127.29.112.190:41859)
0811 18:58:11.541455 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:11.541464 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:11.541972 (+ 508us) server_negotiation.cc:407] Connection header received
0811 18:58:11.542172 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:11.542177 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:11.542255 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:11.542363 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:11.543638 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:11.544535 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:11.546028 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:11.546455 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:11.546617 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:11.550845 (+ 4228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:58:11.550850 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:58:11.550854 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:58:11.550887 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:58:11.553962 (+ 3075us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:11.554772 (+ 810us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:11.554780 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:11.554784 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:11.554963 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:11.555378 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:11.555384 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:11.555387 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:11.555797 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:58:11.556126 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:11.556360 (+ 234us) server_negotiation.cc:299] Negotiation successful
0811 18:58:11.556607 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 18:58:11.561295 11160 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52964:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:58:11.562124 11160 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:58:11.569737 11389 tablet_service.cc:1467] Processing CreateTablet for tablet a8ad2dec90d64b48a88baa46733c141b (DEFAULT_TABLE table=test-table [id=107d80efc9c6438b8860889a769c5814]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:58:11.569911 11614 tablet_service.cc:1467] Processing CreateTablet for tablet a8ad2dec90d64b48a88baa46733c141b (DEFAULT_TABLE table=test-table [id=107d80efc9c6438b8860889a769c5814]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:58:11.570034 11853 tablet_service.cc:1467] Processing CreateTablet for tablet a8ad2dec90d64b48a88baa46733c141b (DEFAULT_TABLE table=test-table [id=107d80efc9c6438b8860889a769c5814]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:58:11.570255 11389 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8ad2dec90d64b48a88baa46733c141b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:58:11.570377 11614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8ad2dec90d64b48a88baa46733c141b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:58:11.570415 11853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8ad2dec90d64b48a88baa46733c141b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:58:11.582165 12136 tablet_bootstrap.cc:492] T a8ad2dec90d64b48a88baa46733c141b P 2134f25fb0da4a129e908b6769b69f76: Bootstrap starting.
I20260811 18:58:11.582996 12135 tablet_bootstrap.cc:492] T a8ad2dec90d64b48a88baa46733c141b P c2d72e507d484000b6768f9a7f376ffb: Bootstrap starting.
I20260811 18:58:11.584156 12136 tablet_bootstrap.cc:654] T a8ad2dec90d64b48a88baa46733c141b P 2134f25fb0da4a129e908b6769b69f76: Neither blocks nor log segments found. Creating new log.
I20260811 18:58:11.584887 12135 tablet_bootstrap.cc:654] T a8ad2dec90d64b48a88baa46733c141b P c2d72e507d484000b6768f9a7f376ffb: Neither blocks nor log segments found. Creating new log.
I20260811 18:58:11.585053 12137 tablet_bootstrap.cc:492] T a8ad2dec90d64b48a88baa46733c141b P 7b050c9777734174b5b4fbc473fef18d: Bootstrap starting.
I20260811 18:58:11.587292 12137 tablet_bootstrap.cc:654] T a8ad2dec90d64b48a88baa46733c141b P 7b050c9777734174b5b4fbc473fef18d: Neither blocks nor log segments found. Creating new log.
I20260811 18:58:11.587427 12135 tablet_bootstrap.cc:492] T a8ad2dec90d64b48a88baa46733c141b P c2d72e507d484000b6768f9a7f376ffb: No bootstrap required, opened a new log
I20260811 18:58:11.587612 12135 ts_tablet_manager.cc:1397] T a8ad2dec90d64b48a88baa46733c141b P c2d72e507d484000b6768f9a7f376ffb: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260811 18:58:11.588508 12135 raft_consensus.cc:348] T a8ad2dec90d64b48a88baa46733c141b P c2d72e507d484000b6768f9a7f376ffb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2d72e507d484000b6768f9a7f376ffb" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 37641 } } peers { permanent_uuid: "7b050c9777734174b5b4fbc473fef18d" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 44157 } } peers { permanent_uuid: "2134f25fb0da4a129e908b6769b69f76" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 37327 } }
I20260811 18:58:11.588792 12135 raft_consensus.cc:374] T a8ad2dec90d64b48a88baa46733c141b P c2d72e507d484000b6768f9a7f376ffb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:58:11.588930 12135 raft_consensus.cc:729] T a8ad2dec90d64b48a88baa46733c141b P c2d72e507d484000b6768f9a7f376ffb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2d72e507d484000b6768f9a7f376ffb, State: Initialized, Role: FOLLOWER
I20260811 18:58:11.589210 12135 consensus_queue.cc:260] T a8ad2dec90d64b48a88baa46733c141b P c2d72e507d484000b6768f9a7f376ffb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2d72e507d484000b6768f9a7f376ffb" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 37641 } } peers { permanent_uuid: "7b050c9777734174b5b4fbc473fef18d" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 44157 } } peers { permanent_uuid: "2134f25fb0da4a129e908b6769b69f76" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 37327 } }
I20260811 18:58:11.590162 12135 ts_tablet_manager.cc:1428] T a8ad2dec90d64b48a88baa46733c141b P c2d72e507d484000b6768f9a7f376ffb: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260811 18:58:11.591682 12136 tablet_bootstrap.cc:492] T a8ad2dec90d64b48a88baa46733c141b P 2134f25fb0da4a129e908b6769b69f76: No bootstrap required, opened a new log
I20260811 18:58:11.591882 12136 ts_tablet_manager.cc:1397] T a8ad2dec90d64b48a88baa46733c141b P 2134f25fb0da4a129e908b6769b69f76: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.002s
I20260811 18:58:11.592672 12136 raft_consensus.cc:348] T a8ad2dec90d64b48a88baa46733c141b P 2134f25fb0da4a129e908b6769b69f76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2d72e507d484000b6768f9a7f376ffb" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 37641 } } peers { permanent_uuid: "7b050c9777734174b5b4fbc473fef18d" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 44157 } } peers { permanent_uuid: "2134f25fb0da4a129e908b6769b69f76" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 37327 } }
I20260811 18:58:11.592978 12136 raft_consensus.cc:374] T a8ad2dec90d64b48a88baa46733c141b P 2134f25fb0da4a129e908b6769b69f76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:58:11.593145 12136 raft_consensus.cc:729] T a8ad2dec90d64b48a88baa46733c141b P 2134f25fb0da4a129e908b6769b69f76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2134f25fb0da4a129e908b6769b69f76, State: Initialized, Role: FOLLOWER
I20260811 18:58:11.594105 12137 tablet_bootstrap.cc:492] T a8ad2dec90d64b48a88baa46733c141b P 7b050c9777734174b5b4fbc473fef18d: No bootstrap required, opened a new log
I20260811 18:58:11.594254 12137 ts_tablet_manager.cc:1397] T a8ad2dec90d64b48a88baa46733c141b P 7b050c9777734174b5b4fbc473fef18d: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.004s
I20260811 18:58:11.595083 12137 raft_consensus.cc:348] T a8ad2dec90d64b48a88baa46733c141b P 7b050c9777734174b5b4fbc473fef18d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2d72e507d484000b6768f9a7f376ffb" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 37641 } } peers { permanent_uuid: "7b050c9777734174b5b4fbc473fef18d" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 44157 } } peers { permanent_uuid: "2134f25fb0da4a129e908b6769b69f76" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 37327 } }
I20260811 18:58:11.595300 12137 raft_consensus.cc:374] T a8ad2dec90d64b48a88baa46733c141b P 7b050c9777734174b5b4fbc473fef18d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:58:11.595446 12137 raft_consensus.cc:729] T a8ad2dec90d64b48a88baa46733c141b P 7b050c9777734174b5b4fbc473fef18d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b050c9777734174b5b4fbc473fef18d, State: Initialized, Role: FOLLOWER
I20260811 18:58:11.595695 12137 consensus_queue.cc:260] T a8ad2dec90d64b48a88baa46733c141b P 7b050c9777734174b5b4fbc473fef18d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2d72e507d484000b6768f9a7f376ffb" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 37641 } } peers { permanent_uuid: "7b050c9777734174b5b4fbc473fef18d" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 44157 } } peers { permanent_uuid: "2134f25fb0da4a129e908b6769b69f76" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 37327 } }
I20260811 18:58:11.596289 12137 ts_tablet_manager.cc:1428] T a8ad2dec90d64b48a88baa46733c141b P 7b050c9777734174b5b4fbc473fef18d: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260811 18:58:11.596587 12136 consensus_queue.cc:260] T a8ad2dec90d64b48a88baa46733c141b P 2134f25fb0da4a129e908b6769b69f76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2d72e507d484000b6768f9a7f376ffb" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 37641 } } peers { permanent_uuid: "7b050c9777734174b5b4fbc473fef18d" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 44157 } } peers { permanent_uuid: "2134f25fb0da4a129e908b6769b69f76" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 37327 } }
I20260811 18:58:11.597186 12136 ts_tablet_manager.cc:1428] T a8ad2dec90d64b48a88baa46733c141b P 2134f25fb0da4a129e908b6769b69f76: Time spent starting tablet: real 0.005s user 0.000s sys 0.001s
I20260811 18:58:11.630759 12103 raft_consensus.cc:482] T a8ad2dec90d64b48a88baa46733c141b P c2d72e507d484000b6768f9a7f376ffb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:58:11.631019 12103 raft_consensus.cc:504] T a8ad2dec90d64b48a88baa46733c141b P c2d72e507d484000b6768f9a7f376ffb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2d72e507d484000b6768f9a7f376ffb" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 37641 } } peers { permanent_uuid: "7b050c9777734174b5b4fbc473fef18d" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 44157 } } peers { permanent_uuid: "2134f25fb0da4a129e908b6769b69f76" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 37327 } }
I20260811 18:58:11.631753 12103 leader_election.cc:288] T a8ad2dec90d64b48a88baa46733c141b P c2d72e507d484000b6768f9a7f376ffb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7b050c9777734174b5b4fbc473fef18d (127.29.112.130:44157), 2134f25fb0da4a129e908b6769b69f76 (127.29.112.131:37327)
I20260811 18:58:11.632290 11660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8ad2dec90d64b48a88baa46733c141b" candidate_uuid: "c2d72e507d484000b6768f9a7f376ffb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b050c9777734174b5b4fbc473fef18d" is_pre_election: true
I20260811 18:58:11.632428 11889 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8ad2dec90d64b48a88baa46733c141b" candidate_uuid: "c2d72e507d484000b6768f9a7f376ffb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2134f25fb0da4a129e908b6769b69f76" is_pre_election: true
I20260811 18:58:11.632576 11660 raft_consensus.cc:2454] T a8ad2dec90d64b48a88baa46733c141b P 7b050c9777734174b5b4fbc473fef18d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2d72e507d484000b6768f9a7f376ffb in term 0.
I20260811 18:58:11.632714 11889 raft_consensus.cc:2454] T a8ad2dec90d64b48a88baa46733c141b P 2134f25fb0da4a129e908b6769b69f76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2d72e507d484000b6768f9a7f376ffb in term 0.
I20260811 18:58:11.633077 11301 leader_election.cc:302] T a8ad2dec90d64b48a88baa46733c141b P c2d72e507d484000b6768f9a7f376ffb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7b050c9777734174b5b4fbc473fef18d, c2d72e507d484000b6768f9a7f376ffb; no voters:
I20260811 18:58:11.633374 12103 raft_consensus.cc:2781] T a8ad2dec90d64b48a88baa46733c141b P c2d72e507d484000b6768f9a7f376ffb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:58:11.633488 12103 raft_consensus.cc:482] T a8ad2dec90d64b48a88baa46733c141b P c2d72e507d484000b6768f9a7f376ffb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:58:11.633630 12103 raft_consensus.cc:3037] T a8ad2dec90d64b48a88baa46733c141b P c2d72e507d484000b6768f9a7f376ffb [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:58:11.636209 12103 raft_consensus.cc:504] T a8ad2dec90d64b48a88baa46733c141b P c2d72e507d484000b6768f9a7f376ffb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2d72e507d484000b6768f9a7f376ffb" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 37641 } } peers { permanent_uuid: "7b050c9777734174b5b4fbc473fef18d" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 44157 } } peers { permanent_uuid: "2134f25fb0da4a129e908b6769b69f76" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 37327 } }
I20260811 18:58:11.636834 12103 leader_election.cc:288] T a8ad2dec90d64b48a88baa46733c141b P c2d72e507d484000b6768f9a7f376ffb [CANDIDATE]: Term 1 election: Requested vote from peers 7b050c9777734174b5b4fbc473fef18d (127.29.112.130:44157), 2134f25fb0da4a129e908b6769b69f76 (127.29.112.131:37327)
I20260811 18:58:11.637283 11660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8ad2dec90d64b48a88baa46733c141b" candidate_uuid: "c2d72e507d484000b6768f9a7f376ffb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b050c9777734174b5b4fbc473fef18d"
I20260811 18:58:11.637358 11889 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8ad2dec90d64b48a88baa46733c141b" candidate_uuid: "c2d72e507d484000b6768f9a7f376ffb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2134f25fb0da4a129e908b6769b69f76"
I20260811 18:58:11.637575 11660 raft_consensus.cc:3037] T a8ad2dec90d64b48a88baa46733c141b P 7b050c9777734174b5b4fbc473fef18d [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:58:11.637615 11889 raft_consensus.cc:3037] T a8ad2dec90d64b48a88baa46733c141b P 2134f25fb0da4a129e908b6769b69f76 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:58:11.640393 11660 raft_consensus.cc:2454] T a8ad2dec90d64b48a88baa46733c141b P 7b050c9777734174b5b4fbc473fef18d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2d72e507d484000b6768f9a7f376ffb in term 1.
I20260811 18:58:11.640820 11301 leader_election.cc:302] T a8ad2dec90d64b48a88baa46733c141b P c2d72e507d484000b6768f9a7f376ffb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7b050c9777734174b5b4fbc473fef18d, c2d72e507d484000b6768f9a7f376ffb; no voters:
I20260811 18:58:11.640839 11889 raft_consensus.cc:2454] T a8ad2dec90d64b48a88baa46733c141b P 2134f25fb0da4a129e908b6769b69f76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2d72e507d484000b6768f9a7f376ffb in term 1.
I20260811 18:58:11.641248 12103 raft_consensus.cc:2781] T a8ad2dec90d64b48a88baa46733c141b P c2d72e507d484000b6768f9a7f376ffb [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:58:11.641444 12103 raft_consensus.cc:686] T a8ad2dec90d64b48a88baa46733c141b P c2d72e507d484000b6768f9a7f376ffb [term 1 LEADER]: Becoming Leader. State: Replica: c2d72e507d484000b6768f9a7f376ffb, State: Running, Role: LEADER
I20260811 18:58:11.641707 12103 consensus_queue.cc:237] T a8ad2dec90d64b48a88baa46733c141b P c2d72e507d484000b6768f9a7f376ffb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2d72e507d484000b6768f9a7f376ffb" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 37641 } } peers { permanent_uuid: "7b050c9777734174b5b4fbc473fef18d" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 44157 } } peers { permanent_uuid: "2134f25fb0da4a129e908b6769b69f76" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 37327 } }
I20260811 18:58:11.644883 11160 catalog_manager.cc:5495] T a8ad2dec90d64b48a88baa46733c141b P c2d72e507d484000b6768f9a7f376ffb reported cstate change: term changed from 0 to 1, leader changed from <none> to c2d72e507d484000b6768f9a7f376ffb (127.29.112.129). New cstate: current_term: 1 leader_uuid: "c2d72e507d484000b6768f9a7f376ffb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2d72e507d484000b6768f9a7f376ffb" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 37641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7b050c9777734174b5b4fbc473fef18d" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 44157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2134f25fb0da4a129e908b6769b69f76" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 37327 } health_report { overall_health: UNKNOWN } } }
I20260811 18:58:11.675769 12071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:11.668969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45442 (local address 127.29.112.129:37641)
0811 18:58:11.669179 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:11.669188 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:11.669315 (+ 127us) server_negotiation.cc:407] Connection header received
0811 18:58:11.669536 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:11.669543 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:11.669658 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:11.669771 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:58:11.670192 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:11.671351 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:11.673430 (+ 2079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:11.674119 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:11.674284 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:11.674899 (+ 615us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:58:11.675059 (+ 160us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:58:11.675165 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:11.675343 (+ 178us) server_negotiation.cc:299] Negotiation successful
0811 18:58:11.675443 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260811 18:58:11.678944 11660 raft_consensus.cc:1261] T a8ad2dec90d64b48a88baa46733c141b P 7b050c9777734174b5b4fbc473fef18d [term 1 FOLLOWER]: Refusing update from remote peer c2d72e507d484000b6768f9a7f376ffb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:58:11.679054 11889 raft_consensus.cc:1261] T a8ad2dec90d64b48a88baa46733c141b P 2134f25fb0da4a129e908b6769b69f76 [term 1 FOLLOWER]: Refusing update from remote peer c2d72e507d484000b6768f9a7f376ffb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:58:11.679615 12109 consensus_queue.cc:1035] T a8ad2dec90d64b48a88baa46733c141b P c2d72e507d484000b6768f9a7f376ffb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7b050c9777734174b5b4fbc473fef18d" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 44157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:58:11.680536 12109 consensus_queue.cc:1035] T a8ad2dec90d64b48a88baa46733c141b P c2d72e507d484000b6768f9a7f376ffb [LEADER]: Connected to new peer: Peer: permanent_uuid: "2134f25fb0da4a129e908b6769b69f76" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 37327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:58:11.694057 12085 mvcc.cc:205] Tried to move back new op lower bound from 7317400337110028288 to 7317400336967065600. Current Snapshot: MvccSnapshot[applied={T|T < 7317400337110028288 or (T in {7317400337110028288})}]
I20260811 18:58:11.701349 11160 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52964:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:58:11.701740 11160 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52964:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:58:11.705736 11160 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 392e71a7a404425489017441bdfbad94: Sending DeleteTablet for 3 replicas of tablet a8ad2dec90d64b48a88baa46733c141b
I20260811 18:58:11.706858 11389 tablet_service.cc:1514] Processing DeleteTablet for tablet a8ad2dec90d64b48a88baa46733c141b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:58:11 UTC) from {username='oryx', principal='oryx/127.29.112.190@KRBTEST.COM'} at 127.0.0.1:45414
I20260811 18:58:11.707204 11614 tablet_service.cc:1514] Processing DeleteTablet for tablet a8ad2dec90d64b48a88baa46733c141b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:58:11 UTC) from {username='oryx', principal='oryx/127.29.112.190@KRBTEST.COM'} at 127.0.0.1:43512
I20260811 18:58:11.707516 12115 tablet_replica.cc:330] T a8ad2dec90d64b48a88baa46733c141b P c2d72e507d484000b6768f9a7f376ffb: stopping tablet replica
I20260811 18:58:11.707728 12115 raft_consensus.cc:2229] T a8ad2dec90d64b48a88baa46733c141b P c2d72e507d484000b6768f9a7f376ffb [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:58:11.708108 12115 raft_consensus.cc:2258] T a8ad2dec90d64b48a88baa46733c141b P c2d72e507d484000b6768f9a7f376ffb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:58:11.707932 11156 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52964:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:58:11.708412 12117 tablet_replica.cc:330] T a8ad2dec90d64b48a88baa46733c141b P 7b050c9777734174b5b4fbc473fef18d: stopping tablet replica
I20260811 18:58:11.708624 12117 raft_consensus.cc:2229] T a8ad2dec90d64b48a88baa46733c141b P 7b050c9777734174b5b4fbc473fef18d [term 1 FOLLOWER]: Raft consensus shutting down.
W20260811 18:58:11.708683 11156 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:58:11.709009 12117 raft_consensus.cc:2258] T a8ad2dec90d64b48a88baa46733c141b P 7b050c9777734174b5b4fbc473fef18d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:58:11.709525 11853 tablet_service.cc:1514] Processing DeleteTablet for tablet a8ad2dec90d64b48a88baa46733c141b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:58:11 UTC) from {username='oryx', principal='oryx/127.29.112.190@KRBTEST.COM'} at 127.0.0.1:50966
I20260811 18:58:11.709801 12116 tablet_replica.cc:330] T a8ad2dec90d64b48a88baa46733c141b P 2134f25fb0da4a129e908b6769b69f76: stopping tablet replica
I20260811 18:58:11.709995 12116 raft_consensus.cc:2229] T a8ad2dec90d64b48a88baa46733c141b P 2134f25fb0da4a129e908b6769b69f76 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:58:11.710220 12116 raft_consensus.cc:2258] T a8ad2dec90d64b48a88baa46733c141b P 2134f25fb0da4a129e908b6769b69f76 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:58:11.714655 12115 ts_tablet_manager.cc:1905] T a8ad2dec90d64b48a88baa46733c141b P c2d72e507d484000b6768f9a7f376ffb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:58:11.717037 11853 tablet_service.cc:1467] Processing CreateTablet for tablet 690d3d62b10340b2b0e1c0853ff17ff2 (DEFAULT_TABLE table=test-table [id=3f39323b60df41499566c4e2c2322c63]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:58:11.717304 11389 tablet_service.cc:1467] Processing CreateTablet for tablet 690d3d62b10340b2b0e1c0853ff17ff2 (DEFAULT_TABLE table=test-table [id=3f39323b60df41499566c4e2c2322c63]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:58:11.717561 11853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 690d3d62b10340b2b0e1c0853ff17ff2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:58:11.717789 11389 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 690d3d62b10340b2b0e1c0853ff17ff2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:58:11.720873 11614 tablet_service.cc:1467] Processing CreateTablet for tablet 690d3d62b10340b2b0e1c0853ff17ff2 (DEFAULT_TABLE table=test-table [id=3f39323b60df41499566c4e2c2322c63]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:58:11.721359 11614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 690d3d62b10340b2b0e1c0853ff17ff2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:58:11.723393 12135 tablet_bootstrap.cc:492] T 690d3d62b10340b2b0e1c0853ff17ff2 P c2d72e507d484000b6768f9a7f376ffb: Bootstrap starting.
I20260811 18:58:11.725553 12135 tablet_bootstrap.cc:654] T 690d3d62b10340b2b0e1c0853ff17ff2 P c2d72e507d484000b6768f9a7f376ffb: Neither blocks nor log segments found. Creating new log.
I20260811 18:58:11.726029 12116 ts_tablet_manager.cc:1905] T a8ad2dec90d64b48a88baa46733c141b P 2134f25fb0da4a129e908b6769b69f76: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:58:11.728174 12137 tablet_bootstrap.cc:492] T 690d3d62b10340b2b0e1c0853ff17ff2 P 7b050c9777734174b5b4fbc473fef18d: Bootstrap starting.
I20260811 18:58:11.730427 12137 tablet_bootstrap.cc:654] T 690d3d62b10340b2b0e1c0853ff17ff2 P 7b050c9777734174b5b4fbc473fef18d: Neither blocks nor log segments found. Creating new log.
I20260811 18:58:11.731978 12117 ts_tablet_manager.cc:1905] T a8ad2dec90d64b48a88baa46733c141b P 7b050c9777734174b5b4fbc473fef18d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:58:11.739728 12117 ts_tablet_manager.cc:1918] T a8ad2dec90d64b48a88baa46733c141b P 7b050c9777734174b5b4fbc473fef18d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:58:11.739902 12117 log.cc:1196] T a8ad2dec90d64b48a88baa46733c141b P 7b050c9777734174b5b4fbc473fef18d: Deleting WAL directory at /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/minicluster-data/ts-1/wal/wals/a8ad2dec90d64b48a88baa46733c141b
I20260811 18:58:11.740365 12117 ts_tablet_manager.cc:1939] T a8ad2dec90d64b48a88baa46733c141b P 7b050c9777734174b5b4fbc473fef18d: Deleting consensus metadata
I20260811 18:58:11.741583 11136 catalog_manager.cc:4841] TS 7b050c9777734174b5b4fbc473fef18d (127.29.112.130:44157): tablet a8ad2dec90d64b48a88baa46733c141b (table test-table [id=107d80efc9c6438b8860889a769c5814]) successfully deleted
I20260811 18:58:11.741675 12137 tablet_bootstrap.cc:492] T 690d3d62b10340b2b0e1c0853ff17ff2 P 7b050c9777734174b5b4fbc473fef18d: No bootstrap required, opened a new log
I20260811 18:58:11.741856 12137 ts_tablet_manager.cc:1397] T 690d3d62b10340b2b0e1c0853ff17ff2 P 7b050c9777734174b5b4fbc473fef18d: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.006s
I20260811 18:58:11.742688 12137 raft_consensus.cc:348] T 690d3d62b10340b2b0e1c0853ff17ff2 P 7b050c9777734174b5b4fbc473fef18d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2d72e507d484000b6768f9a7f376ffb" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 37641 } } peers { permanent_uuid: "7b050c9777734174b5b4fbc473fef18d" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 44157 } } peers { permanent_uuid: "2134f25fb0da4a129e908b6769b69f76" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 37327 } }
I20260811 18:58:11.742959 12137 raft_consensus.cc:374] T 690d3d62b10340b2b0e1c0853ff17ff2 P 7b050c9777734174b5b4fbc473fef18d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:58:11.743044 12137 raft_consensus.cc:729] T 690d3d62b10340b2b0e1c0853ff17ff2 P 7b050c9777734174b5b4fbc473fef18d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b050c9777734174b5b4fbc473fef18d, State: Initialized, Role: FOLLOWER
I20260811 18:58:11.743616 12137 consensus_queue.cc:260] T 690d3d62b10340b2b0e1c0853ff17ff2 P 7b050c9777734174b5b4fbc473fef18d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2d72e507d484000b6768f9a7f376ffb" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 37641 } } peers { permanent_uuid: "7b050c9777734174b5b4fbc473fef18d" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 44157 } } peers { permanent_uuid: "2134f25fb0da4a129e908b6769b69f76" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 37327 } }
I20260811 18:58:11.744269 12137 ts_tablet_manager.cc:1428] T 690d3d62b10340b2b0e1c0853ff17ff2 P 7b050c9777734174b5b4fbc473fef18d: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260811 18:58:11.746369 12115 ts_tablet_manager.cc:1918] T a8ad2dec90d64b48a88baa46733c141b P c2d72e507d484000b6768f9a7f376ffb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:58:11.749171 12116 ts_tablet_manager.cc:1918] T a8ad2dec90d64b48a88baa46733c141b P 2134f25fb0da4a129e908b6769b69f76: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:58:11.749646 12115 log.cc:1196] T a8ad2dec90d64b48a88baa46733c141b P c2d72e507d484000b6768f9a7f376ffb: Deleting WAL directory at /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/minicluster-data/ts-0/wal/wals/a8ad2dec90d64b48a88baa46733c141b
I20260811 18:58:11.748098 12135 tablet_bootstrap.cc:492] T 690d3d62b10340b2b0e1c0853ff17ff2 P c2d72e507d484000b6768f9a7f376ffb: No bootstrap required, opened a new log
I20260811 18:58:11.750258 12116 log.cc:1196] T a8ad2dec90d64b48a88baa46733c141b P 2134f25fb0da4a129e908b6769b69f76: Deleting WAL directory at /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/minicluster-data/ts-2/wal/wals/a8ad2dec90d64b48a88baa46733c141b
I20260811 18:58:11.750267 12135 ts_tablet_manager.cc:1397] T 690d3d62b10340b2b0e1c0853ff17ff2 P c2d72e507d484000b6768f9a7f376ffb: Time spent bootstrapping tablet: real 0.027s user 0.004s sys 0.006s
I20260811 18:58:11.750520 12115 ts_tablet_manager.cc:1939] T a8ad2dec90d64b48a88baa46733c141b P c2d72e507d484000b6768f9a7f376ffb: Deleting consensus metadata
I20260811 18:58:11.749591 12136 tablet_bootstrap.cc:492] T 690d3d62b10340b2b0e1c0853ff17ff2 P 2134f25fb0da4a129e908b6769b69f76: Bootstrap starting.
I20260811 18:58:11.752542 12136 tablet_bootstrap.cc:654] T 690d3d62b10340b2b0e1c0853ff17ff2 P 2134f25fb0da4a129e908b6769b69f76: Neither blocks nor log segments found. Creating new log.
I20260811 18:58:11.751658 12135 raft_consensus.cc:348] T 690d3d62b10340b2b0e1c0853ff17ff2 P c2d72e507d484000b6768f9a7f376ffb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2d72e507d484000b6768f9a7f376ffb" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 37641 } } peers { permanent_uuid: "7b050c9777734174b5b4fbc473fef18d" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 44157 } } peers { permanent_uuid: "2134f25fb0da4a129e908b6769b69f76" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 37327 } }
I20260811 18:58:11.753520 12135 raft_consensus.cc:374] T 690d3d62b10340b2b0e1c0853ff17ff2 P c2d72e507d484000b6768f9a7f376ffb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:58:11.753626 12135 raft_consensus.cc:729] T 690d3d62b10340b2b0e1c0853ff17ff2 P c2d72e507d484000b6768f9a7f376ffb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2d72e507d484000b6768f9a7f376ffb, State: Initialized, Role: FOLLOWER
I20260811 18:58:11.753872 12116 ts_tablet_manager.cc:1939] T a8ad2dec90d64b48a88baa46733c141b P 2134f25fb0da4a129e908b6769b69f76: Deleting consensus metadata
I20260811 18:58:11.753960 12135 consensus_queue.cc:260] T 690d3d62b10340b2b0e1c0853ff17ff2 P c2d72e507d484000b6768f9a7f376ffb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2d72e507d484000b6768f9a7f376ffb" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 37641 } } peers { permanent_uuid: "7b050c9777734174b5b4fbc473fef18d" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 44157 } } peers { permanent_uuid: "2134f25fb0da4a129e908b6769b69f76" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 37327 } }
I20260811 18:58:11.755235 12135 ts_tablet_manager.cc:1428] T 690d3d62b10340b2b0e1c0853ff17ff2 P c2d72e507d484000b6768f9a7f376ffb: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260811 18:58:11.755345 12136 tablet_bootstrap.cc:492] T 690d3d62b10340b2b0e1c0853ff17ff2 P 2134f25fb0da4a129e908b6769b69f76: No bootstrap required, opened a new log
I20260811 18:58:11.755507 12136 ts_tablet_manager.cc:1397] T 690d3d62b10340b2b0e1c0853ff17ff2 P 2134f25fb0da4a129e908b6769b69f76: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.002s
I20260811 18:58:11.756515 12136 raft_consensus.cc:348] T 690d3d62b10340b2b0e1c0853ff17ff2 P 2134f25fb0da4a129e908b6769b69f76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2d72e507d484000b6768f9a7f376ffb" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 37641 } } peers { permanent_uuid: "7b050c9777734174b5b4fbc473fef18d" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 44157 } } peers { permanent_uuid: "2134f25fb0da4a129e908b6769b69f76" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 37327 } }
I20260811 18:58:11.756772 12136 raft_consensus.cc:374] T 690d3d62b10340b2b0e1c0853ff17ff2 P 2134f25fb0da4a129e908b6769b69f76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:58:11.756858 12136 raft_consensus.cc:729] T 690d3d62b10340b2b0e1c0853ff17ff2 P 2134f25fb0da4a129e908b6769b69f76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2134f25fb0da4a129e908b6769b69f76, State: Initialized, Role: FOLLOWER
I20260811 18:58:11.757193 11137 catalog_manager.cc:4841] TS c2d72e507d484000b6768f9a7f376ffb (127.29.112.129:37641): tablet a8ad2dec90d64b48a88baa46733c141b (table test-table [id=107d80efc9c6438b8860889a769c5814]) successfully deleted
I20260811 18:58:11.757149 12136 consensus_queue.cc:260] T 690d3d62b10340b2b0e1c0853ff17ff2 P 2134f25fb0da4a129e908b6769b69f76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2d72e507d484000b6768f9a7f376ffb" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 37641 } } peers { permanent_uuid: "7b050c9777734174b5b4fbc473fef18d" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 44157 } } peers { permanent_uuid: "2134f25fb0da4a129e908b6769b69f76" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 37327 } }
I20260811 18:58:11.757500 11139 catalog_manager.cc:4841] TS 2134f25fb0da4a129e908b6769b69f76 (127.29.112.131:37327): tablet a8ad2dec90d64b48a88baa46733c141b (table test-table [id=107d80efc9c6438b8860889a769c5814]) successfully deleted
I20260811 18:58:11.758279 12136 ts_tablet_manager.cc:1428] T 690d3d62b10340b2b0e1c0853ff17ff2 P 2134f25fb0da4a129e908b6769b69f76: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260811 18:58:11.770495 12009 raft_consensus.cc:482] T 690d3d62b10340b2b0e1c0853ff17ff2 P 2134f25fb0da4a129e908b6769b69f76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:58:11.770763 12009 raft_consensus.cc:504] T 690d3d62b10340b2b0e1c0853ff17ff2 P 2134f25fb0da4a129e908b6769b69f76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2d72e507d484000b6768f9a7f376ffb" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 37641 } } peers { permanent_uuid: "7b050c9777734174b5b4fbc473fef18d" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 44157 } } peers { permanent_uuid: "2134f25fb0da4a129e908b6769b69f76" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 37327 } }
I20260811 18:58:11.772697 12009 leader_election.cc:288] T 690d3d62b10340b2b0e1c0853ff17ff2 P 2134f25fb0da4a129e908b6769b69f76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2d72e507d484000b6768f9a7f376ffb (127.29.112.129:37641), 7b050c9777734174b5b4fbc473fef18d (127.29.112.130:44157)
I20260811 18:58:11.785594 12071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:11.773166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.131:45091 (local address 127.29.112.129:37641)
0811 18:58:11.773378 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:11.773386 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:11.779210 (+ 5824us) server_negotiation.cc:407] Connection header received
0811 18:58:11.779509 (+ 299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:11.779515 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:11.779597 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:11.779699 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:58:11.780260 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:11.781304 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:11.783469 (+ 2165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:11.784414 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:11.784585 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:11.784686 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:11.785162 (+ 476us) server_negotiation.cc:299] Negotiation successful
0811 18:58:11.785299 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 18:58:11.785691 12155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:11.772815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:37641 (local address 127.29.112.131:45091)
0811 18:58:11.779084 (+ 6269us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:11.779114 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:11.779348 (+ 234us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:11.779714 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:11.779721 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:11.779751 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:58:11.780087 (+ 336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:11.780101 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:11.781477 (+ 1376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:11.781484 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:11.783250 (+ 1766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:11.783265 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:11.784856 (+ 1591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:11.784865 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:11.785003 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:11.785026 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 18:58:11.785135 (+ 109us) client_negotiation.cc:240] Negotiation successful
0811 18:58:11.785280 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6103,"thread_start_us":6380,"threads_started":1}
I20260811 18:58:11.786311 12079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:11.773642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.131:38287 (local address 127.29.112.130:44157)
0811 18:58:11.773944 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:11.773950 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:11.779608 (+ 5658us) server_negotiation.cc:407] Connection header received
0811 18:58:11.779820 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:11.779825 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:11.779914 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:11.779988 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:58:11.780688 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:11.781477 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:11.783910 (+ 2433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:11.785284 (+ 1374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:11.785443 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:11.785523 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:11.785930 (+ 407us) server_negotiation.cc:299] Negotiation successful
0811 18:58:11.786072 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114}
I20260811 18:58:11.786324 11425 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "690d3d62b10340b2b0e1c0853ff17ff2" candidate_uuid: "2134f25fb0da4a129e908b6769b69f76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2d72e507d484000b6768f9a7f376ffb" is_pre_election: true
I20260811 18:58:11.786675 11425 raft_consensus.cc:2454] T 690d3d62b10340b2b0e1c0853ff17ff2 P c2d72e507d484000b6768f9a7f376ffb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2134f25fb0da4a129e908b6769b69f76 in term 0.
I20260811 18:58:11.787065 12153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:11.773195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.130:44157 (local address 127.29.112.131:38287)
0811 18:58:11.779495 (+ 6300us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:11.779512 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:11.779642 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:11.780205 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:11.780209 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:11.780237 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:58:11.780527 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:11.780534 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:11.781637 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:11.781643 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:11.783719 (+ 2076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:11.783734 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:11.785507 (+ 1773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:11.785513 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:11.785720 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:11.785761 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 18:58:11.786520 (+ 759us) client_negotiation.cc:240] Negotiation successful
0811 18:58:11.786674 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6174,"thread_start_us":5562,"threads_started":1}
I20260811 18:58:11.787360 11770 leader_election.cc:302] T 690d3d62b10340b2b0e1c0853ff17ff2 P 2134f25fb0da4a129e908b6769b69f76 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2134f25fb0da4a129e908b6769b69f76, c2d72e507d484000b6768f9a7f376ffb; no voters:
I20260811 18:58:11.787808 12009 raft_consensus.cc:2781] T 690d3d62b10340b2b0e1c0853ff17ff2 P 2134f25fb0da4a129e908b6769b69f76 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:58:11.787714 11660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "690d3d62b10340b2b0e1c0853ff17ff2" candidate_uuid: "2134f25fb0da4a129e908b6769b69f76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b050c9777734174b5b4fbc473fef18d" is_pre_election: true
I20260811 18:58:11.787962 12009 raft_consensus.cc:482] T 690d3d62b10340b2b0e1c0853ff17ff2 P 2134f25fb0da4a129e908b6769b69f76 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:58:11.788017 11660 raft_consensus.cc:2454] T 690d3d62b10340b2b0e1c0853ff17ff2 P 7b050c9777734174b5b4fbc473fef18d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2134f25fb0da4a129e908b6769b69f76 in term 0.
I20260811 18:58:11.788103 12009 raft_consensus.cc:3037] T 690d3d62b10340b2b0e1c0853ff17ff2 P 2134f25fb0da4a129e908b6769b69f76 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:58:11.790340 12009 raft_consensus.cc:504] T 690d3d62b10340b2b0e1c0853ff17ff2 P 2134f25fb0da4a129e908b6769b69f76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2d72e507d484000b6768f9a7f376ffb" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 37641 } } peers { permanent_uuid: "7b050c9777734174b5b4fbc473fef18d" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 44157 } } peers { permanent_uuid: "2134f25fb0da4a129e908b6769b69f76" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 37327 } }
I20260811 18:58:11.790988 12009 leader_election.cc:288] T 690d3d62b10340b2b0e1c0853ff17ff2 P 2134f25fb0da4a129e908b6769b69f76 [CANDIDATE]: Term 1 election: Requested vote from peers c2d72e507d484000b6768f9a7f376ffb (127.29.112.129:37641), 7b050c9777734174b5b4fbc473fef18d (127.29.112.130:44157)
I20260811 18:58:11.791553 11425 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "690d3d62b10340b2b0e1c0853ff17ff2" candidate_uuid: "2134f25fb0da4a129e908b6769b69f76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2d72e507d484000b6768f9a7f376ffb"
I20260811 18:58:11.791564 11660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "690d3d62b10340b2b0e1c0853ff17ff2" candidate_uuid: "2134f25fb0da4a129e908b6769b69f76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b050c9777734174b5b4fbc473fef18d"
I20260811 18:58:11.791810 11660 raft_consensus.cc:3037] T 690d3d62b10340b2b0e1c0853ff17ff2 P 7b050c9777734174b5b4fbc473fef18d [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:58:11.791810 11425 raft_consensus.cc:3037] T 690d3d62b10340b2b0e1c0853ff17ff2 P c2d72e507d484000b6768f9a7f376ffb [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:58:11.794839 11425 raft_consensus.cc:2454] T 690d3d62b10340b2b0e1c0853ff17ff2 P c2d72e507d484000b6768f9a7f376ffb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2134f25fb0da4a129e908b6769b69f76 in term 1.
I20260811 18:58:11.794822 11660 raft_consensus.cc:2454] T 690d3d62b10340b2b0e1c0853ff17ff2 P 7b050c9777734174b5b4fbc473fef18d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2134f25fb0da4a129e908b6769b69f76 in term 1.
I20260811 18:58:11.795253 11770 leader_election.cc:302] T 690d3d62b10340b2b0e1c0853ff17ff2 P 2134f25fb0da4a129e908b6769b69f76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2134f25fb0da4a129e908b6769b69f76, c2d72e507d484000b6768f9a7f376ffb; no voters:
I20260811 18:58:11.795646 12009 raft_consensus.cc:2781] T 690d3d62b10340b2b0e1c0853ff17ff2 P 2134f25fb0da4a129e908b6769b69f76 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:58:11.796200 12009 raft_consensus.cc:686] T 690d3d62b10340b2b0e1c0853ff17ff2 P 2134f25fb0da4a129e908b6769b69f76 [term 1 LEADER]: Becoming Leader. State: Replica: 2134f25fb0da4a129e908b6769b69f76, State: Running, Role: LEADER
I20260811 18:58:11.796653 12009 consensus_queue.cc:237] T 690d3d62b10340b2b0e1c0853ff17ff2 P 2134f25fb0da4a129e908b6769b69f76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2d72e507d484000b6768f9a7f376ffb" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 37641 } } peers { permanent_uuid: "7b050c9777734174b5b4fbc473fef18d" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 44157 } } peers { permanent_uuid: "2134f25fb0da4a129e908b6769b69f76" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 37327 } }
I20260811 18:58:11.801033 11161 catalog_manager.cc:5495] T 690d3d62b10340b2b0e1c0853ff17ff2 P 2134f25fb0da4a129e908b6769b69f76 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2134f25fb0da4a129e908b6769b69f76 (127.29.112.131). New cstate: current_term: 1 leader_uuid: "2134f25fb0da4a129e908b6769b69f76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2d72e507d484000b6768f9a7f376ffb" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 37641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7b050c9777734174b5b4fbc473fef18d" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 44157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2134f25fb0da4a129e908b6769b69f76" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 37327 } health_report { overall_health: HEALTHY } } }
I20260811 18:58:11.835162 12161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:11.828966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50974 (local address 127.29.112.131:37327)
0811 18:58:11.829562 (+ 596us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:11.829567 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:11.829585 (+ 18us) server_negotiation.cc:407] Connection header received
0811 18:58:11.829666 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:11.829670 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:11.829742 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:11.829940 (+ 198us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:58:11.830408 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:11.831509 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:11.832834 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:11.833243 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:11.833399 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:11.833879 (+ 480us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:58:11.834457 (+ 578us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:58:11.834653 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:11.834733 (+ 80us) server_negotiation.cc:299] Negotiation successful
0811 18:58:11.834826 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":493,"thread_start_us":336,"threads_started":1}
I20260811 18:58:11.850611 12079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:11.845197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.131:49551 (local address 127.29.112.130:44157)
0811 18:58:11.845365 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:11.845372 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:11.845944 (+ 572us) server_negotiation.cc:407] Connection header received
0811 18:58:11.846060 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:11.846064 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:11.846131 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:11.846213 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:58:11.846747 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:11.847465 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:11.848793 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:11.849291 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:11.849391 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:11.849884 (+ 493us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:58:11.850009 (+ 125us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:58:11.850084 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:11.850284 (+ 200us) server_negotiation.cc:299] Negotiation successful
0811 18:58:11.850352 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 18:58:11.850723 12167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:11.845126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.130:44157 (local address 127.29.112.131:49551)
0811 18:58:11.845833 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:11.845849 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:11.845968 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:11.846244 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:11.846247 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:11.846256 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 18:58:11.846603 (+ 347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:11.846612 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:11.847584 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:11.847588 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:11.848651 (+ 1063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:11.848663 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:11.849485 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:11.849490 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:11.849614 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:11.849668 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 18:58:11.850128 (+ 460us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 18:58:11.850139 (+ 11us) client_negotiation.cc:769] Sending connection context
0811 18:58:11.850218 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 18:58:11.850321 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":367,"threads_started":1}
I20260811 18:58:11.860265 11425 raft_consensus.cc:1261] T 690d3d62b10340b2b0e1c0853ff17ff2 P c2d72e507d484000b6768f9a7f376ffb [term 1 FOLLOWER]: Refusing update from remote peer 2134f25fb0da4a129e908b6769b69f76: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:58:11.860229 11660 raft_consensus.cc:1261] T 690d3d62b10340b2b0e1c0853ff17ff2 P 7b050c9777734174b5b4fbc473fef18d [term 1 FOLLOWER]: Refusing update from remote peer 2134f25fb0da4a129e908b6769b69f76: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:58:11.861054 12158 consensus_queue.cc:1035] T 690d3d62b10340b2b0e1c0853ff17ff2 P 2134f25fb0da4a129e908b6769b69f76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7b050c9777734174b5b4fbc473fef18d" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 44157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:58:11.861629 12009 consensus_queue.cc:1035] T 690d3d62b10340b2b0e1c0853ff17ff2 P 2134f25fb0da4a129e908b6769b69f76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2d72e507d484000b6768f9a7f376ffb" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 37641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:58:11.890851 12161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:11.885393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.130:39149 (local address 127.29.112.131:37327)
0811 18:58:11.885554 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:11.885561 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:11.885640 (+ 79us) server_negotiation.cc:407] Connection header received
0811 18:58:11.885794 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:11.885797 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:11.885861 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:11.885971 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:58:11.886344 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:11.887186 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:11.888490 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:11.888911 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:11.889044 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:11.889438 (+ 394us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:58:11.889566 (+ 128us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:58:11.889907 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:11.890129 (+ 222us) server_negotiation.cc:299] Negotiation successful
0811 18:58:11.890296 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 18:58:11.890909 12112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:11.885262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.131:37327 (local address 127.29.112.130:39149)
0811 18:58:11.885541 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:11.885562 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:11.885682 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:11.885995 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:11.886000 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:11.886014 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 18:58:11.886212 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:11.886220 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:11.887354 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:11.887359 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:11.888335 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:11.888346 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:11.889055 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:11.889061 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:11.889224 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:11.889260 (+ 36us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 18:58:11.889904 (+ 644us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 18:58:11.889926 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 18:58:11.890076 (+ 150us) client_negotiation.cc:240] Negotiation successful
0811 18:58:11.890322 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138}
I20260811 18:58:11.921409 11156 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52964:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:58:11.921658 11156 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52964:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:58:11.924796 11156 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 392e71a7a404425489017441bdfbad94: Sending DeleteTablet for 3 replicas of tablet 690d3d62b10340b2b0e1c0853ff17ff2
I20260811 18:58:11.925655 11614 tablet_service.cc:1514] Processing DeleteTablet for tablet 690d3d62b10340b2b0e1c0853ff17ff2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:58:11 UTC) from {username='oryx', principal='oryx/127.29.112.190@KRBTEST.COM'} at 127.0.0.1:43512
I20260811 18:58:11.925627 11389 tablet_service.cc:1514] Processing DeleteTablet for tablet 690d3d62b10340b2b0e1c0853ff17ff2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:58:11 UTC) from {username='oryx', principal='oryx/127.29.112.190@KRBTEST.COM'} at 127.0.0.1:45414
I20260811 18:58:11.925971 12117 tablet_replica.cc:330] T 690d3d62b10340b2b0e1c0853ff17ff2 P 7b050c9777734174b5b4fbc473fef18d: stopping tablet replica
I20260811 18:58:11.925967 12115 tablet_replica.cc:330] T 690d3d62b10340b2b0e1c0853ff17ff2 P c2d72e507d484000b6768f9a7f376ffb: stopping tablet replica
I20260811 18:58:11.926168 12115 raft_consensus.cc:2229] T 690d3d62b10340b2b0e1c0853ff17ff2 P c2d72e507d484000b6768f9a7f376ffb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:58:11.926177 12117 raft_consensus.cc:2229] T 690d3d62b10340b2b0e1c0853ff17ff2 P 7b050c9777734174b5b4fbc473fef18d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:58:11.926401 12115 raft_consensus.cc:2258] T 690d3d62b10340b2b0e1c0853ff17ff2 P c2d72e507d484000b6768f9a7f376ffb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:58:11.926409 12117 raft_consensus.cc:2258] T 690d3d62b10340b2b0e1c0853ff17ff2 P 7b050c9777734174b5b4fbc473fef18d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:58:11.926720 11853 tablet_service.cc:1514] Processing DeleteTablet for tablet 690d3d62b10340b2b0e1c0853ff17ff2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:58:11 UTC) from {username='oryx', principal='oryx/127.29.112.190@KRBTEST.COM'} at 127.0.0.1:50966
I20260811 18:58:11.927062 12116 tablet_replica.cc:330] T 690d3d62b10340b2b0e1c0853ff17ff2 P 2134f25fb0da4a129e908b6769b69f76: stopping tablet replica
I20260811 18:58:11.927259 12116 raft_consensus.cc:2229] T 690d3d62b10340b2b0e1c0853ff17ff2 P 2134f25fb0da4a129e908b6769b69f76 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:58:11.927809 12116 raft_consensus.cc:2258] T 690d3d62b10340b2b0e1c0853ff17ff2 P 2134f25fb0da4a129e908b6769b69f76 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:58:11.927834 12115 ts_tablet_manager.cc:1905] T 690d3d62b10340b2b0e1c0853ff17ff2 P c2d72e507d484000b6768f9a7f376ffb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:58:11.927980 12117 ts_tablet_manager.cc:1905] T 690d3d62b10340b2b0e1c0853ff17ff2 P 7b050c9777734174b5b4fbc473fef18d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:58:11.928260 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 11274
I20260811 18:58:11.928922 12116 ts_tablet_manager.cc:1905] T 690d3d62b10340b2b0e1c0853ff17ff2 P 2134f25fb0da4a129e908b6769b69f76: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:58:11.933637 12115 ts_tablet_manager.cc:1918] T 690d3d62b10340b2b0e1c0853ff17ff2 P c2d72e507d484000b6768f9a7f376ffb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 18:58:11.933794 12115 log.cc:1196] T 690d3d62b10340b2b0e1c0853ff17ff2 P c2d72e507d484000b6768f9a7f376ffb: Deleting WAL directory at /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/minicluster-data/ts-0/wal/wals/690d3d62b10340b2b0e1c0853ff17ff2
I20260811 18:58:11.934280 12115 ts_tablet_manager.cc:1939] T 690d3d62b10340b2b0e1c0853ff17ff2 P c2d72e507d484000b6768f9a7f376ffb: Deleting consensus metadata
I20260811 18:58:11.935477 11137 catalog_manager.cc:4841] TS c2d72e507d484000b6768f9a7f376ffb (127.29.112.129:37641): tablet 690d3d62b10340b2b0e1c0853ff17ff2 (table test-table [id=3f39323b60df41499566c4e2c2322c63]) successfully deleted
I20260811 18:58:11.936508 12116 ts_tablet_manager.cc:1918] T 690d3d62b10340b2b0e1c0853ff17ff2 P 2134f25fb0da4a129e908b6769b69f76: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 18:58:11.936671 12116 log.cc:1196] T 690d3d62b10340b2b0e1c0853ff17ff2 P 2134f25fb0da4a129e908b6769b69f76: Deleting WAL directory at /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/minicluster-data/ts-2/wal/wals/690d3d62b10340b2b0e1c0853ff17ff2
I20260811 18:58:11.937242 12116 ts_tablet_manager.cc:1939] T 690d3d62b10340b2b0e1c0853ff17ff2 P 2134f25fb0da4a129e908b6769b69f76: Deleting consensus metadata
I20260811 18:58:11.938520 11139 catalog_manager.cc:4841] TS 2134f25fb0da4a129e908b6769b69f76 (127.29.112.131:37327): tablet 690d3d62b10340b2b0e1c0853ff17ff2 (table test-table [id=3f39323b60df41499566c4e2c2322c63]) successfully deleted
I20260811 18:58:11.939085 12117 ts_tablet_manager.cc:1918] T 690d3d62b10340b2b0e1c0853ff17ff2 P 7b050c9777734174b5b4fbc473fef18d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 18:58:11.939235 12117 log.cc:1196] T 690d3d62b10340b2b0e1c0853ff17ff2 P 7b050c9777734174b5b4fbc473fef18d: Deleting WAL directory at /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474615205622-30146-0/minicluster-data/ts-1/wal/wals/690d3d62b10340b2b0e1c0853ff17ff2
I20260811 18:58:11.939618 12117 ts_tablet_manager.cc:1939] T 690d3d62b10340b2b0e1c0853ff17ff2 P 7b050c9777734174b5b4fbc473fef18d: Deleting consensus metadata
I20260811 18:58:11.940516 11136 catalog_manager.cc:4841] TS 7b050c9777734174b5b4fbc473fef18d (127.29.112.130:44157): tablet 690d3d62b10340b2b0e1c0853ff17ff2 (table test-table [id=3f39323b60df41499566c4e2c2322c63]) successfully deleted
Aug 11 18:58:11 dist-test-slave-h6n0 krb5kdc[11077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474691, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.29.112.129@KRBTEST.COM
I20260811 18:58:11.962273 12071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:11.930969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45448 (local address 127.29.112.129:37641)
0811 18:58:11.931131 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:11.931138 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:11.944699 (+ 13561us) server_negotiation.cc:407] Connection header received
0811 18:58:11.944928 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:11.944935 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:11.945048 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:11.945220 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:11.946816 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:11.947996 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:11.949264 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:11.949897 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:11.950035 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:11.955214 (+ 5179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:58:11.955235 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:58:11.955255 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:58:11.955302 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:58:11.959366 (+ 4064us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:11.960099 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:11.960110 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:11.960116 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:11.960278 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:11.960651 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:11.960657 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:11.960661 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:11.961120 (+ 459us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:58:11.961468 (+ 348us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:11.961667 (+ 199us) server_negotiation.cc:299] Negotiation successful
0811 18:58:11.961900 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 18:58:11.963055 11495 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:58:12.128364 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 11274
I20260811 18:58:12.161139 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 11510
Aug 11 18:58:12 dist-test-slave-h6n0 krb5kdc[11077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474691, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.29.112.130@KRBTEST.COM
I20260811 18:58:12.178695 12079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:12.161960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43532 (local address 127.29.112.130:44157)
0811 18:58:12.162147 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:12.162153 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:12.162176 (+ 23us) server_negotiation.cc:407] Connection header received
0811 18:58:12.162308 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:12.162315 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:12.162418 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:12.162552 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:12.163803 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:12.164830 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:12.166115 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:12.166823 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:12.166948 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:12.171490 (+ 4542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:58:12.171508 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:58:12.171525 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:58:12.171575 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:58:12.175586 (+ 4011us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:12.176294 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:12.176303 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:12.176306 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:12.176468 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:12.176903 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:12.176911 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:12.176914 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:12.177400 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:58:12.177700 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:12.178011 (+ 311us) server_negotiation.cc:299] Negotiation successful
0811 18:58:12.178259 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 18:58:12.179455 11718 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:58:12.319607 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 11510
I20260811 18:58:12.354933 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 11738
Aug 11 18:58:12 dist-test-slave-h6n0 krb5kdc[11077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474691, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.29.112.131@KRBTEST.COM
I20260811 18:58:12.377441 12161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:12.355873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50978 (local address 127.29.112.131:37327)
0811 18:58:12.356095 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:12.356102 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:12.356127 (+ 25us) server_negotiation.cc:407] Connection header received
0811 18:58:12.356339 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:12.356345 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:12.356465 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:12.356598 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:12.358250 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:12.359617 (+ 1367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:12.361501 (+ 1884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:12.362172 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:12.362302 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:12.368945 (+ 6643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:58:12.368972 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:58:12.368995 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:58:12.369064 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:58:12.373506 (+ 4442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:12.374609 (+ 1103us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:12.374618 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:12.374621 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:12.374829 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:12.375341 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:12.375349 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:12.375353 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:12.376101 (+ 748us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:58:12.376385 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:12.376810 (+ 425us) server_negotiation.cc:299] Negotiation successful
0811 18:58:12.377054 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260811 18:58:12.378346 11957 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:58:12.532470 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 11738
I20260811 18:58:12.561430 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 11099
I20260811 18:58:12.563009 11226 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:58:12.684563 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 11099
2026-08-11T18:58:12Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (6418 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786474615205622-30146-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:58:12 dist-test-slave-h6n0 krb5kdc[12187](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:58:12 dist-test-slave-h6n0 krb5kdc[12187](info): set up 2 sockets
Aug 11 18:58:12 dist-test-slave-h6n0 krb5kdc[12187](info): commencing operation
krb5kdc: starting...
W20260811 18:58:14.070459 30146 mini_kdc.cc:121] Time spent starting KDC: real 1.331s user 0.001s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786474615205622-30146-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786474615205622-30146-0/krb5kdc/test-user.keytab.
Aug 11 18:58:14 dist-test-slave-h6n0 krb5kdc[12187](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474694, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:58:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:58:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.112.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab.
Entry for principal kudu/127.29.112.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab.
I20260811 18:58:14.358079 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786474615205622-30146-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786474615205622-30146-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786474615205622-30146-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.190:33185
--webserver_interface=127.29.112.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab
--principal=kudu/127.29.112.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:43923
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.190:33185
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:58:14.701218 12208 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:58:14.701689 12208 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:58:14.701859 12208 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:58:14.709156 12208 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:58:14.709307 12208 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:58:14.709419 12208 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:58:14.709525 12208 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:58:14.709621 12208 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:58:14.719520 12208 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:43923
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786474615205622-30146-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786474615205622-30146-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.112.190:33185
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.190:33185
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.112.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786474615205622-30146-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:58:14.720856 12208 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:58:14.722188 12208 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:58:14.733748 12216 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:14.733803 12219 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:14.733834 12217 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:58:14.734928 12208 server_base.cc:1034] running on GCE node
I20260811 18:58:14.735635 12208 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:58:14.736930 12208 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:58:14.738135 12208 hybrid_clock.cc:648] HybridClock initialized: now 1786474694738082 us; error 48 us; skew 500 ppm
I20260811 18:58:14.738423 12208 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-11T18:58:14Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (2118 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:58:14 dist-test-slave-h6n0 krb5kdc[12231](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:58:14 dist-test-slave-h6n0 krb5kdc[12231](info): set up 2 sockets
Aug 11 18:58:14 dist-test-slave-h6n0 krb5kdc[12231](info): commencing operation
krb5kdc: starting...
W20260811 18:58:16.178363 30146 mini_kdc.cc:121] Time spent starting KDC: real 1.320s user 0.003s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/krb5kdc/test-user.keytab.
Aug 11 18:58:16 dist-test-slave-h6n0 krb5kdc[12231](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474696, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:58:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:58:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.112.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab.
Entry for principal kudu/127.29.112.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab.
I20260811 18:58:16.488837 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.190:44265
--webserver_interface=127.29.112.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab
--principal=kudu/127.29.112.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:44253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.190:44265
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:58:16.841724 12251 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:58:16.842029 12251 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:58:16.842121 12251 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:58:16.850951 12251 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:58:16.851125 12251 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:58:16.851181 12251 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:58:16.851250 12251 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:58:16.851303 12251 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:58:16.862159 12251 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:44253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.112.190:44265
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab
--principal=kudu/127.29.112.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.190:44265
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.112.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:58:16.863642 12251 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:58:16.865001 12251 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:58:16.877794 12258 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:16.877880 12262 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:16.877918 12260 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:58:16.884646 12251 server_base.cc:1034] running on GCE node
I20260811 18:58:16.885627 12251 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:58:16.887254 12251 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:58:16.888535 12251 hybrid_clock.cc:648] HybridClock initialized: now 1786474696888486 us; error 58 us; skew 500 ppm
I20260811 18:58:16.888922 12251 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:58:16 dist-test-slave-h6n0 krb5kdc[12231](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474696, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:58:16.893047 12251 init.cc:377] Logged in from keytab as kudu/127.29.112.190@KRBTEST.COM (short username kudu)
I20260811 18:58:16.894860 12251 webserver.cc:460] Webserver started at http://127.29.112.190:42077/ using document root <none> and password file <none>
I20260811 18:58:16.895602 12251 fs_manager.cc:359] Metadata directory not provided
I20260811 18:58:16.895721 12251 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:58:16.896016 12251 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:58:16.899296 12251 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/master-0/data/instance:
uuid: "9b9be5c218854d9bb009fd94d113d44c"
format_stamp: "Formatted at 2026-08-11 18:58:16 on dist-test-slave-h6n0"
server_key: "f78fbac7699007b0656e866ed5726430"
server_key_iv: "e084e137382a02e777e6f62c1f865c6e"
server_key_version: "encryptionkey@0"
I20260811 18:58:16.900107 12251 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/master-0/wal/instance:
uuid: "9b9be5c218854d9bb009fd94d113d44c"
format_stamp: "Formatted at 2026-08-11 18:58:16 on dist-test-slave-h6n0"
server_key: "f78fbac7699007b0656e866ed5726430"
server_key_iv: "e084e137382a02e777e6f62c1f865c6e"
server_key_version: "encryptionkey@0"
I20260811 18:58:16.905989 12251 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 18:58:16.909974 12274 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:16.911510 12251 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:58:16.911718 12251 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/master-0/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/master-0/wal
uuid: "9b9be5c218854d9bb009fd94d113d44c"
format_stamp: "Formatted at 2026-08-11 18:58:16 on dist-test-slave-h6n0"
server_key: "f78fbac7699007b0656e866ed5726430"
server_key_iv: "e084e137382a02e777e6f62c1f865c6e"
server_key_version: "encryptionkey@0"
I20260811 18:58:16.911960 12251 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:58:16.928705 12251 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:58:16.933715 12251 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:58:16.934291 12251 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:58:16.960421 12251 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.190:44265
I20260811 18:58:16.960541 12366 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.190:44265 every 8 connection(s)
I20260811 18:58:16.961978 12251 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
I20260811 18:58:16.965957 12367 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:58:16.969559 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 12251
I20260811 18:58:16.969807 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/master-0/wal/instance
I20260811 18:58:16.970475 30146 external_mini_cluster.cc:1428] Setting key dda590ed43ba2d9a4f44ac44ff584e1a
I20260811 18:58:16.979813 12367 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b9be5c218854d9bb009fd94d113d44c: Bootstrap starting.
I20260811 18:58:16.983902 12367 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9b9be5c218854d9bb009fd94d113d44c: Neither blocks nor log segments found. Creating new log.
I20260811 18:58:16.985337 12367 log.cc:824] T 00000000000000000000000000000000 P 9b9be5c218854d9bb009fd94d113d44c: Log is configured to *not* fsync() on all Append() calls
Aug 11 18:58:16 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474696, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 18:58:16.988727 12367 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b9be5c218854d9bb009fd94d113d44c: No bootstrap required, opened a new log
I20260811 18:58:16.996114 12370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:16.972505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55172 (local address 127.29.112.190:44265)
0811 18:58:16.973392 (+ 887us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:16.973404 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:16.973452 (+ 48us) server_negotiation.cc:407] Connection header received
0811 18:58:16.975154 (+ 1702us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:16.975174 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:16.975561 (+ 387us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:16.975872 (+ 311us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:16.977966 (+ 2094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:16.979294 (+ 1328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:16.981212 (+ 1918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:16.981916 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:16.982068 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:16.987316 (+ 5248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:58:16.987326 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:58:16.987344 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:58:16.987390 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:58:16.991465 (+ 4075us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:16.992137 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:16.992150 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:16.992160 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:16.992383 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:16.992733 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:16.992738 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:16.992741 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:16.993503 (+ 762us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:58:16.993803 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:16.994174 (+ 371us) server_negotiation.cc:299] Negotiation successful
0811 18:58:16.994683 (+ 509us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":666,"thread_start_us":441,"threads_started":1}
I20260811 18:58:16.999529 12367 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9b9be5c218854d9bb009fd94d113d44c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b9be5c218854d9bb009fd94d113d44c" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 44265 } }
I20260811 18:58:16.999868 12367 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9b9be5c218854d9bb009fd94d113d44c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:58:16.999938 12367 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9b9be5c218854d9bb009fd94d113d44c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b9be5c218854d9bb009fd94d113d44c, State: Initialized, Role: FOLLOWER
I20260811 18:58:17.000506 12367 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9b9be5c218854d9bb009fd94d113d44c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b9be5c218854d9bb009fd94d113d44c" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 44265 } }
I20260811 18:58:17.000715 12367 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9b9be5c218854d9bb009fd94d113d44c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:58:17.000847 12367 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9b9be5c218854d9bb009fd94d113d44c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:58:17.001014 12367 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9b9be5c218854d9bb009fd94d113d44c [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:58:17.003556 12367 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9b9be5c218854d9bb009fd94d113d44c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b9be5c218854d9bb009fd94d113d44c" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 44265 } }
I20260811 18:58:17.004029 12367 leader_election.cc:302] T 00000000000000000000000000000000 P 9b9be5c218854d9bb009fd94d113d44c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9b9be5c218854d9bb009fd94d113d44c; no voters:
I20260811 18:58:17.004571 12367 leader_election.cc:288] T 00000000000000000000000000000000 P 9b9be5c218854d9bb009fd94d113d44c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:58:17.004755 12373 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9b9be5c218854d9bb009fd94d113d44c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:58:17.005900 12373 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9b9be5c218854d9bb009fd94d113d44c [term 1 LEADER]: Becoming Leader. State: Replica: 9b9be5c218854d9bb009fd94d113d44c, State: Running, Role: LEADER
I20260811 18:58:17.006430 12373 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9b9be5c218854d9bb009fd94d113d44c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b9be5c218854d9bb009fd94d113d44c" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 44265 } }
I20260811 18:58:17.007234 12367 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9b9be5c218854d9bb009fd94d113d44c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:58:17.009797 12374 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b9be5c218854d9bb009fd94d113d44c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9b9be5c218854d9bb009fd94d113d44c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b9be5c218854d9bb009fd94d113d44c" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 44265 } } }
I20260811 18:58:17.010212 12374 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b9be5c218854d9bb009fd94d113d44c [sys.catalog]: This master's current role is: LEADER
I20260811 18:58:17.009671 12376 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b9be5c218854d9bb009fd94d113d44c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9b9be5c218854d9bb009fd94d113d44c. Latest consensus state: current_term: 1 leader_uuid: "9b9be5c218854d9bb009fd94d113d44c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b9be5c218854d9bb009fd94d113d44c" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 44265 } } }
I20260811 18:58:17.010768 12376 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b9be5c218854d9bb009fd94d113d44c [sys.catalog]: This master's current role is: LEADER
I20260811 18:58:17.012319 12381 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:58:17.017674 12381 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:58:17.027716 12381 catalog_manager.cc:1347] Generated new cluster ID: c431c9ab69d5495390be4cd913c2c7a9
I20260811 18:58:17.027874 12381 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:58:17.039795 12381 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:58:17.042109 12381 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:58:17.049096 12381 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9b9be5c218854d9bb009fd94d113d44c: Generated new TSK 0
I20260811 18:58:17.049974 12381 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.112.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.129.keytab.
Entry for principal kudu/127.29.112.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.129.keytab.
I20260811 18:58:17.132689 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.129:0
--local_ip_for_outbound_sockets=127.29.112.129
--webserver_interface=127.29.112.129
--webserver_port=0
--tserver_master_addrs=127.29.112.190:44265
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.129.keytab
--principal=kudu/127.29.112.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:44253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:58:17.513010 12409 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:58:17.513299 12409 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:58:17.513478 12409 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:58:17.521341 12409 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:58:17.521466 12409 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:58:17.521646 12409 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.129
I20260811 18:58:17.531638 12409 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:44253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.129.keytab
--principal=kudu/127.29.112.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.112.129
--webserver_port=0
--tserver_master_addrs=127.29.112.190:44265
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.129
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:58:17.532987 12409 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:58:17.534320 12409 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:58:17.548722 12419 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:17.548877 12417 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:17.548911 12416 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:58:17.550761 12409 server_base.cc:1034] running on GCE node
I20260811 18:58:17.551390 12409 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:58:17.552346 12409 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:58:17.553596 12409 hybrid_clock.cc:648] HybridClock initialized: now 1786474697553545 us; error 47 us; skew 500 ppm
I20260811 18:58:17.553947 12409 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:58:17 dist-test-slave-h6n0 krb5kdc[12231](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474697, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:58:17.557417 12409 init.cc:377] Logged in from keytab as kudu/127.29.112.129@KRBTEST.COM (short username kudu)
I20260811 18:58:17.559188 12409 webserver.cc:460] Webserver started at http://127.29.112.129:43507/ using document root <none> and password file <none>
I20260811 18:58:17.559929 12409 fs_manager.cc:359] Metadata directory not provided
I20260811 18:58:17.560081 12409 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:58:17.560406 12409 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:58:17.563408 12409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/ts-0/data/instance:
uuid: "6c32213a6c5a4c2fafa2c1e7001e01e4"
format_stamp: "Formatted at 2026-08-11 18:58:17 on dist-test-slave-h6n0"
server_key: "bf7f86a721e459e70ea12465a877d1b3"
server_key_iv: "77a2d28f6fcf234d58b89665184e3f5c"
server_key_version: "encryptionkey@0"
I20260811 18:58:17.564204 12409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/ts-0/wal/instance:
uuid: "6c32213a6c5a4c2fafa2c1e7001e01e4"
format_stamp: "Formatted at 2026-08-11 18:58:17 on dist-test-slave-h6n0"
server_key: "bf7f86a721e459e70ea12465a877d1b3"
server_key_iv: "77a2d28f6fcf234d58b89665184e3f5c"
server_key_version: "encryptionkey@0"
I20260811 18:58:17.569373 12409 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 18:58:17.572805 12432 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:17.574090 12409 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 18:58:17.574368 12409 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/ts-0/wal
uuid: "6c32213a6c5a4c2fafa2c1e7001e01e4"
format_stamp: "Formatted at 2026-08-11 18:58:17 on dist-test-slave-h6n0"
server_key: "bf7f86a721e459e70ea12465a877d1b3"
server_key_iv: "77a2d28f6fcf234d58b89665184e3f5c"
server_key_version: "encryptionkey@0"
I20260811 18:58:17.574606 12409 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:58:17.595283 12409 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:58:17.599809 12409 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:58:17.600152 12409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:58:17.601138 12409 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:58:17.602500 12409 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:58:17.602633 12409 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:17.602788 12409 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:58:17.602864 12409 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:17.639201 12409 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.129:45315
I20260811 18:58:17.639292 12610 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.129:45315 every 8 connection(s)
I20260811 18:58:17.640523 12409 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/ts-0/data/info.pb
I20260811 18:58:17.648128 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 12409
I20260811 18:58:17.648408 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/ts-0/wal/instance
I20260811 18:58:17.649173 30146 external_mini_cluster.cc:1428] Setting key 9555ac8d0bce73cd248b0e4f825dfb99
Aug 11 18:58:17 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474697, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.129@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 18:58:17.663642 12614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:17.643324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.129:45189 (local address 127.29.112.190:44265)
0811 18:58:17.643816 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:17.643820 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:17.644831 (+ 1011us) server_negotiation.cc:407] Connection header received
0811 18:58:17.647314 (+ 2483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:17.647319 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:17.647409 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:17.647529 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:17.649636 (+ 2107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:17.650606 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:17.652282 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:17.652652 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:17.652785 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:17.656812 (+ 4027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:58:17.656820 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:58:17.656827 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:58:17.656868 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:58:17.660222 (+ 3354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:17.661161 (+ 939us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:17.661168 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:17.661173 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:17.661313 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:17.661843 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:17.661851 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:17.661855 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:17.662129 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:58:17.662296 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:17.663034 (+ 738us) server_negotiation.cc:299] Negotiation successful
0811 18:58:17.663258 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376,"thread_start_us":268,"threads_started":1}
I20260811 18:58:17.664532 12615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:17.643709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:44265 (local address 127.29.112.129:45189)
0811 18:58:17.644669 (+ 960us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:17.644717 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:17.647049 (+ 2332us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:17.647802 (+ 753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:17.647813 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:17.648680 (+ 867us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:58:17.649454 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:17.649473 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:17.650756 (+ 1283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:17.650761 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:17.652126 (+ 1365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:17.652143 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:17.652875 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:17.652882 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:17.653109 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:17.653813 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:58:17.653834 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:58:17.656572 (+ 2738us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:58:17.660370 (+ 3798us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:58:17.660381 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:58:17.660398 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:58:17.661040 (+ 642us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:58:17.661431 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:58:17.661438 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:58:17.661444 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:58:17.661706 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:58:17.662276 (+ 570us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:58:17.662283 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:58:17.662660 (+ 377us) client_negotiation.cc:769] Sending connection context
0811 18:58:17.662941 (+ 281us) client_negotiation.cc:240] Negotiation successful
0811 18:58:17.663444 (+ 503us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":716,"thread_start_us":580,"threads_started":1}
I20260811 18:58:17.666054 12611 heartbeater.cc:347] Connected to a master server at 127.29.112.190:44265
I20260811 18:58:17.666383 12611 heartbeater.cc:464] Registering TS with master...
I20260811 18:58:17.667296 12611 heartbeater.cc:511] Master 127.29.112.190:44265 requested a full tablet report, sending...
I20260811 18:58:17.669456 12301 ts_manager.cc:194] Registered new tserver with Master: 6c32213a6c5a4c2fafa2c1e7001e01e4 (127.29.112.129:45315)
I20260811 18:58:17.670953 12301 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.129@KRBTEST.COM'} at 127.29.112.129:45189
WARNING: no policy specified for kudu/127.29.112.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.130.keytab.
Entry for principal kudu/127.29.112.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.130.keytab.
I20260811 18:58:17.712908 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.130:0
--local_ip_for_outbound_sockets=127.29.112.130
--webserver_interface=127.29.112.130
--webserver_port=0
--tserver_master_addrs=127.29.112.190:44265
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.130.keytab
--principal=kudu/127.29.112.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:44253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:58:18.053767 12618 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:58:18.054068 12618 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:58:18.054241 12618 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:58:18.062248 12618 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:58:18.062386 12618 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:58:18.062628 12618 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.130
I20260811 18:58:18.073910 12618 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:44253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.130.keytab
--principal=kudu/127.29.112.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.112.130
--webserver_port=0
--tserver_master_addrs=127.29.112.190:44265
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.130
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:58:18.075361 12618 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:58:18.076663 12618 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:58:18.091296 12627 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:18.091233 12634 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:18.091199 12628 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:58:18.092315 12618 server_base.cc:1034] running on GCE node
I20260811 18:58:18.092963 12618 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:58:18.093963 12618 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:58:18.095223 12618 hybrid_clock.cc:648] HybridClock initialized: now 1786474698095157 us; error 55 us; skew 500 ppm
I20260811 18:58:18.095582 12618 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:58:18 dist-test-slave-h6n0 krb5kdc[12231](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474698, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:58:18.099424 12618 init.cc:377] Logged in from keytab as kudu/127.29.112.130@KRBTEST.COM (short username kudu)
I20260811 18:58:18.101068 12618 webserver.cc:460] Webserver started at http://127.29.112.130:36381/ using document root <none> and password file <none>
I20260811 18:58:18.101784 12618 fs_manager.cc:359] Metadata directory not provided
I20260811 18:58:18.101925 12618 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:58:18.102205 12618 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:58:18.105441 12618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/ts-1/data/instance:
uuid: "2d6953d91b2b4b77b994b7ff6e421ae7"
format_stamp: "Formatted at 2026-08-11 18:58:18 on dist-test-slave-h6n0"
server_key: "c26c8d9e18a1580da924ef03e2c75a4c"
server_key_iv: "1228eee75b189f90f80ac127865e38d6"
server_key_version: "encryptionkey@0"
I20260811 18:58:18.106376 12618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/ts-1/wal/instance:
uuid: "2d6953d91b2b4b77b994b7ff6e421ae7"
format_stamp: "Formatted at 2026-08-11 18:58:18 on dist-test-slave-h6n0"
server_key: "c26c8d9e18a1580da924ef03e2c75a4c"
server_key_iv: "1228eee75b189f90f80ac127865e38d6"
server_key_version: "encryptionkey@0"
I20260811 18:58:18.112257 12618 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260811 18:58:18.116209 12643 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:18.117837 12618 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 18:58:18.118019 12618 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/ts-1/wal
uuid: "2d6953d91b2b4b77b994b7ff6e421ae7"
format_stamp: "Formatted at 2026-08-11 18:58:18 on dist-test-slave-h6n0"
server_key: "c26c8d9e18a1580da924ef03e2c75a4c"
server_key_iv: "1228eee75b189f90f80ac127865e38d6"
server_key_version: "encryptionkey@0"
I20260811 18:58:18.118235 12618 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:58:18.138964 12618 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:58:18.143908 12618 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:58:18.144232 12618 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:58:18.145454 12618 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:58:18.147104 12618 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:58:18.147203 12618 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:18.147384 12618 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:58:18.147442 12618 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:18.186391 12618 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.130:43559
I20260811 18:58:18.186440 12822 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.130:43559 every 8 connection(s)
I20260811 18:58:18.187742 12618 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/ts-1/data/info.pb
I20260811 18:58:18.192065 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 12618
I20260811 18:58:18.192286 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/ts-1/wal/instance
I20260811 18:58:18.192878 30146 external_mini_cluster.cc:1428] Setting key e846a7b4328b7227830ec529c8ed7066
Aug 11 18:58:18 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474698, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.130@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 18:58:18.211711 12826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:18.190493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.130:47511 (local address 127.29.112.190:44265)
0811 18:58:18.191036 (+ 543us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:18.191041 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:18.191971 (+ 930us) server_negotiation.cc:407] Connection header received
0811 18:58:18.194468 (+ 2497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:18.194474 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:18.194585 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:18.194707 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:18.196872 (+ 2165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:18.197596 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:18.200110 (+ 2514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:18.200553 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:18.200652 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:18.205330 (+ 4678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:58:18.205334 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:58:18.205337 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:58:18.205370 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:58:18.208439 (+ 3069us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:18.209207 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:18.209212 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:18.209216 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:18.209358 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:18.209815 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:18.209820 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:18.209822 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:18.210093 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:58:18.210284 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:18.211085 (+ 801us) server_negotiation.cc:299] Negotiation successful
0811 18:58:18.211316 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":441,"thread_start_us":285,"threads_started":1}
I20260811 18:58:18.212792 12829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:18.190956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:44265 (local address 127.29.112.130:47511)
0811 18:58:18.191740 (+ 784us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:18.191811 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:18.194181 (+ 2370us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:18.194994 (+ 813us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:18.195003 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:18.195800 (+ 797us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:58:18.196670 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:18.196699 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:18.197770 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:18.197778 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:18.199947 (+ 2169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:18.199966 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:18.200795 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:18.200804 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:18.201142 (+ 338us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:18.201943 (+ 801us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:58:18.201983 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:58:18.205077 (+ 3094us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:58:18.208600 (+ 3523us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:58:18.208607 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:58:18.208619 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:58:18.209068 (+ 449us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:58:18.209482 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:58:18.209486 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:58:18.209490 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:58:18.209683 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:58:18.210240 (+ 557us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:58:18.210250 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:58:18.210682 (+ 432us) client_negotiation.cc:769] Sending connection context
0811 18:58:18.210961 (+ 279us) client_negotiation.cc:240] Negotiation successful
0811 18:58:18.211503 (+ 542us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":287,"threads_started":1}
I20260811 18:58:18.214481 12824 heartbeater.cc:347] Connected to a master server at 127.29.112.190:44265
I20260811 18:58:18.214860 12824 heartbeater.cc:464] Registering TS with master...
I20260811 18:58:18.215601 12824 heartbeater.cc:511] Master 127.29.112.190:44265 requested a full tablet report, sending...
I20260811 18:58:18.217204 12301 ts_manager.cc:194] Registered new tserver with Master: 2d6953d91b2b4b77b994b7ff6e421ae7 (127.29.112.130:43559)
I20260811 18:58:18.218422 12301 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.130@KRBTEST.COM'} at 127.29.112.130:47511
WARNING: no policy specified for kudu/127.29.112.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.131.keytab.
Entry for principal kudu/127.29.112.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.131.keytab.
I20260811 18:58:18.264475 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.131:0
--local_ip_for_outbound_sockets=127.29.112.131
--webserver_interface=127.29.112.131
--webserver_port=0
--tserver_master_addrs=127.29.112.190:44265
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.131.keytab
--principal=kudu/127.29.112.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:44253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:58:18.633682 12834 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:58:18.633991 12834 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:58:18.634166 12834 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:58:18.641605 12834 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:58:18.641772 12834 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:58:18.641963 12834 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.131
I20260811 18:58:18.652673 12834 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:44253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.131.keytab
--principal=kudu/127.29.112.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.112.131
--webserver_port=0
--tserver_master_addrs=127.29.112.190:44265
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.131
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:58:18.653930 12834 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:58:18.655360 12834 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:58:18.669071 12841 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:18.669031 12842 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:18.668934 12844 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:58:18.670614 12834 server_base.cc:1034] running on GCE node
I20260811 18:58:18.671244 12834 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:58:18.672406 12834 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:58:18.673696 12834 hybrid_clock.cc:648] HybridClock initialized: now 1786474698673650 us; error 34 us; skew 500 ppm
I20260811 18:58:18.674063 12834 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 18:58:18.674692 12611 heartbeater.cc:503] Master 127.29.112.190:44265 was elected leader, sending a full tablet report...
Aug 11 18:58:18 dist-test-slave-h6n0 krb5kdc[12231](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474698, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:58:18.677893 12834 init.cc:377] Logged in from keytab as kudu/127.29.112.131@KRBTEST.COM (short username kudu)
I20260811 18:58:18.679589 12834 webserver.cc:460] Webserver started at http://127.29.112.131:40177/ using document root <none> and password file <none>
I20260811 18:58:18.680320 12834 fs_manager.cc:359] Metadata directory not provided
I20260811 18:58:18.680491 12834 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:58:18.680891 12834 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:58:18.684051 12834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/ts-2/data/instance:
uuid: "1988db1b14a242b5b586b76f340bcdfb"
format_stamp: "Formatted at 2026-08-11 18:58:18 on dist-test-slave-h6n0"
server_key: "44fdf3772b9574957929b4018ef77a6d"
server_key_iv: "ad13cb19c4ce5c6b378a07d2c36cb18a"
server_key_version: "encryptionkey@0"
I20260811 18:58:18.684911 12834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/ts-2/wal/instance:
uuid: "1988db1b14a242b5b586b76f340bcdfb"
format_stamp: "Formatted at 2026-08-11 18:58:18 on dist-test-slave-h6n0"
server_key: "44fdf3772b9574957929b4018ef77a6d"
server_key_iv: "ad13cb19c4ce5c6b378a07d2c36cb18a"
server_key_version: "encryptionkey@0"
I20260811 18:58:18.690691 12834 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260811 18:58:18.694098 12860 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:18.695346 12834 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:58:18.695521 12834 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/ts-2/wal
uuid: "1988db1b14a242b5b586b76f340bcdfb"
format_stamp: "Formatted at 2026-08-11 18:58:18 on dist-test-slave-h6n0"
server_key: "44fdf3772b9574957929b4018ef77a6d"
server_key_iv: "ad13cb19c4ce5c6b378a07d2c36cb18a"
server_key_version: "encryptionkey@0"
I20260811 18:58:18.695741 12834 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:58:18.751032 12834 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:58:18.755820 12834 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:58:18.756138 12834 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:58:18.757372 12834 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:58:18.759006 12834 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:58:18.759119 12834 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:18.759270 12834 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:58:18.759351 12834 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:18.795686 12834 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.131:39027
I20260811 18:58:18.795727 13043 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.131:39027 every 8 connection(s)
I20260811 18:58:18.797449 12834 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/ts-2/data/info.pb
I20260811 18:58:18.803361 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 12834
I20260811 18:58:18.803581 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/ts-2/wal/instance
I20260811 18:58:18.804049 30146 external_mini_cluster.cc:1428] Setting key 6ed7d95d01bf5ebf53039e2ba4dd5047
Aug 11 18:58:18 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474698, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.131@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 18:58:18.820772 13046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:18.800435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.131:38117 (local address 127.29.112.190:44265)
0811 18:58:18.800881 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:18.800886 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:18.801901 (+ 1015us) server_negotiation.cc:407] Connection header received
0811 18:58:18.803890 (+ 1989us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:18.803896 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:18.803991 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:18.804116 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:18.805878 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:18.806839 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:18.808671 (+ 1832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:18.809253 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:18.809359 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:18.813463 (+ 4104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:58:18.813468 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:58:18.813471 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:58:18.813503 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:58:18.817286 (+ 3783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:18.818159 (+ 873us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:18.818167 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:18.818170 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:18.818299 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:18.818813 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:18.818898 (+ 85us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:18.818901 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:18.819309 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:58:18.819486 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:18.820170 (+ 684us) server_negotiation.cc:299] Negotiation successful
0811 18:58:18.820388 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":199,"threads_started":1}
I20260811 18:58:18.821766 13047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:18.800989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:44265 (local address 127.29.112.131:38117)
0811 18:58:18.801721 (+ 732us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:18.801772 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:18.803588 (+ 1816us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:18.804353 (+ 765us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:18.804360 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:18.805021 (+ 661us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:58:18.805667 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:18.805686 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:18.807059 (+ 1373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:18.807065 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:18.808508 (+ 1443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:18.808520 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:18.809468 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:18.809473 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:18.809690 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:18.810410 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:58:18.810442 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:58:18.813235 (+ 2793us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:58:18.817454 (+ 4219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:58:18.817463 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:58:18.817478 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:58:18.818031 (+ 553us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:58:18.818417 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:58:18.818422 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:58:18.818426 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:58:18.818704 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:58:18.819458 (+ 754us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:58:18.819467 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:58:18.819803 (+ 336us) client_negotiation.cc:769] Sending connection context
0811 18:58:18.820070 (+ 267us) client_negotiation.cc:240] Negotiation successful
0811 18:58:18.820604 (+ 534us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":296,"threads_started":1}
I20260811 18:58:18.823252 13044 heartbeater.cc:347] Connected to a master server at 127.29.112.190:44265
I20260811 18:58:18.823975 13044 heartbeater.cc:464] Registering TS with master...
I20260811 18:58:18.825076 13044 heartbeater.cc:511] Master 127.29.112.190:44265 requested a full tablet report, sending...
I20260811 18:58:18.826910 12301 ts_manager.cc:194] Registered new tserver with Master: 1988db1b14a242b5b586b76f340bcdfb (127.29.112.131:39027)
I20260811 18:58:18.827798 12301 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.131@KRBTEST.COM'} at 127.29.112.131:38117
I20260811 18:58:18.832852 30146 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.29.112.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.112.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.112.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/krb5kdc/oryx_127.29.112.129.keytab.
Entry for principal oryx/127.29.112.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/krb5kdc/oryx_127.29.112.129.keytab.
WARNING: no policy specified for oryx/127.29.112.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.112.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.112.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/krb5kdc/oryx_127.29.112.130.keytab.
Entry for principal oryx/127.29.112.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/krb5kdc/oryx_127.29.112.130.keytab.
WARNING: no policy specified for oryx/127.29.112.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.112.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.112.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/krb5kdc/oryx_127.29.112.131.keytab.
Entry for principal oryx/127.29.112.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/krb5kdc/oryx_127.29.112.131.keytab.
I20260811 18:58:18.999140 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 12409
Aug 11 18:58:19 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474696, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.129@KRBTEST.COM
I20260811 18:58:19.015659 13060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:19.000395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59934 (local address 127.29.112.129:45315)
0811 18:58:19.001047 (+ 652us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:19.001055 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:19.001085 (+ 30us) server_negotiation.cc:407] Connection header received
0811 18:58:19.001149 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:19.001155 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:19.001352 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:19.001503 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:19.002818 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:19.003814 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:19.005019 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:19.005559 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:19.005654 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:19.009892 (+ 4238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:58:19.009910 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:58:19.009922 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:58:19.009960 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:58:19.012893 (+ 2933us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:19.013555 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:19.013562 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:19.013564 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:19.013733 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:19.014155 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:19.014159 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:19.014162 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:19.014507 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:58:19.014810 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:19.015160 (+ 350us) server_negotiation.cc:299] Negotiation successful
0811 18:58:19.015359 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":490,"thread_start_us":344,"threads_started":1}
I20260811 18:58:19.016919 12603 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:58:19.105077 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 12409
I20260811 18:58:19.134052 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 12618
Aug 11 18:58:19 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474696, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.130@KRBTEST.COM
I20260811 18:58:19.154495 13063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:19.135694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42722 (local address 127.29.112.130:43559)
0811 18:58:19.136501 (+ 807us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:19.136509 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:19.136540 (+ 31us) server_negotiation.cc:407] Connection header received
0811 18:58:19.136611 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:19.136617 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:19.136925 (+ 308us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:19.137096 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:19.138607 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:19.139650 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:19.141035 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:19.141639 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:19.141761 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:19.147034 (+ 5273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:58:19.147047 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:58:19.147061 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:58:19.147105 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:58:19.150599 (+ 3494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:19.151854 (+ 1255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:19.151866 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:19.151869 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:19.152162 (+ 293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:19.152604 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:19.152609 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:19.152611 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:19.153136 (+ 525us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:58:19.153515 (+ 379us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:19.153865 (+ 350us) server_negotiation.cc:299] Negotiation successful
0811 18:58:19.154134 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":637,"thread_start_us":492,"threads_started":1}
I20260811 18:58:19.156584 12816 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:58:19.244030 12824 heartbeater.cc:503] Master 127.29.112.190:44265 was elected leader, sending a full tablet report...
I20260811 18:58:19.245519 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 12618
W20260811 18:58:19.274438 12279 connection.cc:535] server connection from 127.29.112.130:47511 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 18:58:19.274830 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 12834
Aug 11 18:58:19 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474696, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.131@KRBTEST.COM
I20260811 18:58:19.291060 13065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:19.275867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47508 (local address 127.29.112.131:39027)
0811 18:58:19.276394 (+ 527us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:19.276401 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:19.276431 (+ 30us) server_negotiation.cc:407] Connection header received
0811 18:58:19.276489 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:19.276495 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:19.276700 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:19.276881 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:19.278235 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:19.279261 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:19.280522 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:19.281059 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:19.281202 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:19.285580 (+ 4378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:58:19.285591 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:58:19.285603 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:58:19.285644 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:58:19.288552 (+ 2908us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:19.289172 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:19.289179 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:19.289181 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:19.289291 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:19.289669 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:19.289672 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:19.289674 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:19.289976 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:58:19.290215 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:19.290442 (+ 227us) server_negotiation.cc:299] Negotiation successful
0811 18:58:19.290764 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":229,"threads_started":1}
I20260811 18:58:19.292305 13039 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:58:19.378046 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 12834
I20260811 18:58:19.401860 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 12251
I20260811 18:58:19.403127 12357 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:58:19.508140 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 12251
I20260811 18:58:19.536916 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.190:44265
--webserver_interface=127.29.112.190
--webserver_port=42077
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab
--principal=kudu/127.29.112.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:44253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.190:44265
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:58:19.895509 13074 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:58:19.895849 13074 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:58:19.896028 13074 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:58:19.903329 13074 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:58:19.903559 13074 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:58:19.903853 13074 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:58:19.904098 13074 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:58:19.904189 13074 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:58:19.914144 13074 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:44253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.112.190:44265
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab
--principal=kudu/127.29.112.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.190:44265
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.112.190
--webserver_port=42077
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:58:19.915539 13074 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:58:19.916823 13074 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:58:19.929191 13084 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:19.929409 13085 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:19.929643 13088 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:58:19.931015 13074 server_base.cc:1034] running on GCE node
I20260811 18:58:19.931908 13074 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:58:19.933640 13074 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:58:19.934897 13074 hybrid_clock.cc:648] HybridClock initialized: now 1786474699934836 us; error 47 us; skew 500 ppm
I20260811 18:58:19.935322 13074 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:58:19 dist-test-slave-h6n0 krb5kdc[12231](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474699, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:58:19.942961 13074 init.cc:377] Logged in from keytab as kudu/127.29.112.190@KRBTEST.COM (short username kudu)
I20260811 18:58:19.944922 13074 webserver.cc:460] Webserver started at http://127.29.112.190:42077/ using document root <none> and password file <none>
I20260811 18:58:19.945706 13074 fs_manager.cc:359] Metadata directory not provided
I20260811 18:58:19.945889 13074 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:58:19.952153 13074 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.002s
I20260811 18:58:19.956210 13101 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:19.957756 13074 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260811 18:58:19.957965 13074 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/master-0/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/master-0/wal
uuid: "9b9be5c218854d9bb009fd94d113d44c"
format_stamp: "Formatted at 2026-08-11 18:58:16 on dist-test-slave-h6n0"
server_key: "f78fbac7699007b0656e866ed5726430"
server_key_iv: "e084e137382a02e777e6f62c1f865c6e"
server_key_version: "encryptionkey@0"
I20260811 18:58:19.959096 13074 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:58:19.983394 13074 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:58:19.988579 13074 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:58:19.988876 13074 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:58:20.008839 13074 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.190:44265
I20260811 18:58:20.008876 13180 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.190:44265 every 8 connection(s)
I20260811 18:58:20.010321 13074 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
I20260811 18:58:20.017596 13181 sys_catalog.cc:263] Verifying existing consensus state
I20260811 18:58:20.017625 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 13074
I20260811 18:58:20.018623 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.129:45315
--local_ip_for_outbound_sockets=127.29.112.129
--tserver_master_addrs=127.29.112.190:44265
--webserver_port=43507
--webserver_interface=127.29.112.129
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.129.keytab
--principal=kudu/127.29.112.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:44253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.29.112.129
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/krb5kdc/oryx_127.29.112.129.keytab with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 18:58:20.022696 13181 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b9be5c218854d9bb009fd94d113d44c: Bootstrap starting.
I20260811 18:58:20.044620 13181 log.cc:824] T 00000000000000000000000000000000 P 9b9be5c218854d9bb009fd94d113d44c: Log is configured to *not* fsync() on all Append() calls
I20260811 18:58:20.054628 13181 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b9be5c218854d9bb009fd94d113d44c: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 18:58:20.055212 13181 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b9be5c218854d9bb009fd94d113d44c: Bootstrap complete.
I20260811 18:58:20.067297 13181 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9b9be5c218854d9bb009fd94d113d44c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b9be5c218854d9bb009fd94d113d44c" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 44265 } }
I20260811 18:58:20.068161 13181 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9b9be5c218854d9bb009fd94d113d44c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b9be5c218854d9bb009fd94d113d44c, State: Initialized, Role: FOLLOWER
I20260811 18:58:20.068809 13181 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9b9be5c218854d9bb009fd94d113d44c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b9be5c218854d9bb009fd94d113d44c" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 44265 } }
I20260811 18:58:20.069053 13181 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9b9be5c218854d9bb009fd94d113d44c [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:58:20.069200 13181 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9b9be5c218854d9bb009fd94d113d44c [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:58:20.069399 13181 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9b9be5c218854d9bb009fd94d113d44c [term 1 FOLLOWER]: Advancing to term 2
I20260811 18:58:20.073248 13181 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9b9be5c218854d9bb009fd94d113d44c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b9be5c218854d9bb009fd94d113d44c" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 44265 } }
I20260811 18:58:20.073783 13181 leader_election.cc:302] T 00000000000000000000000000000000 P 9b9be5c218854d9bb009fd94d113d44c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9b9be5c218854d9bb009fd94d113d44c; no voters:
I20260811 18:58:20.074434 13181 leader_election.cc:288] T 00000000000000000000000000000000 P 9b9be5c218854d9bb009fd94d113d44c [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 18:58:20.074600 13187 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9b9be5c218854d9bb009fd94d113d44c [term 2 FOLLOWER]: Leader election won for term 2
I20260811 18:58:20.075851 13187 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9b9be5c218854d9bb009fd94d113d44c [term 2 LEADER]: Becoming Leader. State: Replica: 9b9be5c218854d9bb009fd94d113d44c, State: Running, Role: LEADER
I20260811 18:58:20.076470 13187 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9b9be5c218854d9bb009fd94d113d44c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b9be5c218854d9bb009fd94d113d44c" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 44265 } }
I20260811 18:58:20.076728 13181 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9b9be5c218854d9bb009fd94d113d44c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:58:20.079721 13191 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b9be5c218854d9bb009fd94d113d44c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9b9be5c218854d9bb009fd94d113d44c. Latest consensus state: current_term: 2 leader_uuid: "9b9be5c218854d9bb009fd94d113d44c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b9be5c218854d9bb009fd94d113d44c" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 44265 } } }
I20260811 18:58:20.079767 13188 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b9be5c218854d9bb009fd94d113d44c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "9b9be5c218854d9bb009fd94d113d44c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b9be5c218854d9bb009fd94d113d44c" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 44265 } } }
I20260811 18:58:20.080575 13188 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b9be5c218854d9bb009fd94d113d44c [sys.catalog]: This master's current role is: LEADER
I20260811 18:58:20.081163 13191 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b9be5c218854d9bb009fd94d113d44c [sys.catalog]: This master's current role is: LEADER
I20260811 18:58:20.095732 13200 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:58:20.101274 13200 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:58:20.102818 13200 catalog_manager.cc:1259] Loaded cluster ID: c431c9ab69d5495390be4cd913c2c7a9
I20260811 18:58:20.102993 13200 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:58:20.107076 13200 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:58:20.109118 13200 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 9b9be5c218854d9bb009fd94d113d44c: Loaded TSK: 0
I20260811 18:58:20.110913 13200 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 18:58:20.389482 13183 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:58:20.389881 13183 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:58:20.390064 13183 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:58:20.398398 13183 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:58:20.398582 13183 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:58:20.398805 13183 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.129
I20260811 18:58:20.408950 13183 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:44253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/krb5kdc/oryx_127.29.112.129.keytab
--principal=oryx/127.29.112.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.129:45315
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.112.129
--webserver_port=43507
--tserver_master_addrs=127.29.112.190:44265
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.129
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:58:20.410285 13183 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:58:20.411648 13183 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:58:20.425112 13225 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:20.425093 13231 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:20.425431 13223 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:58:20.427369 13183 server_base.cc:1034] running on GCE node
I20260811 18:58:20.428113 13183 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:58:20.429160 13183 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:58:20.430384 13183 hybrid_clock.cc:648] HybridClock initialized: now 1786474700430318 us; error 49 us; skew 500 ppm
I20260811 18:58:20.430819 13183 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:58:20 dist-test-slave-h6n0 krb5kdc[12231](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474700, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.112.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:58:20.435243 13183 init.cc:377] Logged in from keytab as oryx/127.29.112.129@KRBTEST.COM (short username oryx)
I20260811 18:58:20.437072 13183 webserver.cc:460] Webserver started at http://127.29.112.129:43507/ using document root <none> and password file <none>
I20260811 18:58:20.437924 13183 fs_manager.cc:359] Metadata directory not provided
I20260811 18:58:20.438088 13183 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:58:20.445101 13183 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 18:58:20.448402 13247 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:20.450098 13183 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260811 18:58:20.450265 13183 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/ts-0/wal
uuid: "6c32213a6c5a4c2fafa2c1e7001e01e4"
format_stamp: "Formatted at 2026-08-11 18:58:17 on dist-test-slave-h6n0"
server_key: "bf7f86a721e459e70ea12465a877d1b3"
server_key_iv: "77a2d28f6fcf234d58b89665184e3f5c"
server_key_version: "encryptionkey@0"
I20260811 18:58:20.451566 13183 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:58:20.478057 13183 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:58:20.482211 13183 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:58:20.482447 13183 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:58:20.483491 13183 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:58:20.484869 13183 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:58:20.484963 13183 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:20.485034 13183 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:58:20.485090 13183 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:20.521790 13183 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.129:45315
I20260811 18:58:20.521893 13418 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.129:45315 every 8 connection(s)
I20260811 18:58:20.523442 13183 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/ts-0/data/info.pb
I20260811 18:58:20.529520 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 13183
I20260811 18:58:20.530292 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.130:43559
--local_ip_for_outbound_sockets=127.29.112.130
--tserver_master_addrs=127.29.112.190:44265
--webserver_port=36381
--webserver_interface=127.29.112.130
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.130.keytab
--principal=kudu/127.29.112.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:44253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.29.112.130
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/krb5kdc/oryx_127.29.112.130.keytab with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 18:58:20 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.129@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:58:20 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.129@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:58:20.545989 13430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:20.526945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:44265 (local address 127.29.112.129:35113)
0811 18:58:20.527807 (+ 862us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:20.527855 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:20.530129 (+ 2274us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:20.531644 (+ 1515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:20.531658 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:20.532490 (+ 832us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:58:20.533263 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:20.533289 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:20.534983 (+ 1694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:20.534991 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:20.537142 (+ 2151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:20.537161 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:20.538374 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:20.538382 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:20.538716 (+ 334us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:20.539513 (+ 797us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:58:20.539548 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:58:20.544555 (+ 5007us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.190:44265: Server oryx/127.29.112.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":586,"thread_start_us":378,"threads_started":1}
W20260811 18:58:20.546202 13426 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:58:20.526821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.129:35113 (local address 127.29.112.190:44265)
0811 18:58:20.527729 (+ 908us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:20.527745 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:20.527976 (+ 231us) server_negotiation.cc:407] Connection header received
0811 18:58:20.530773 (+ 2797us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:20.530786 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:20.531099 (+ 313us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:20.531407 (+ 308us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:20.533479 (+ 2072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:20.534811 (+ 1332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:20.537309 (+ 2498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:20.538111 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:20.538266 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:20.544862 (+ 6596us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.129:35113: BlockingRecv error: recv got EOF from 127.29.112.129:35113 (error 108)
Metrics: {"server-negotiator.queue_time_us":644,"thread_start_us":365,"threads_started":1}
W20260811 18:58:20.546833 13419 heartbeater.cc:644] Failed to heartbeat to 127.29.112.190:44265 (0 consecutive failures): Not authorized: Failed to ping master at 127.29.112.190:44265: Client connection negotiation failed: client connection to 127.29.112.190:44265: Server oryx/127.29.112.190@KRBTEST.COM not found in Kerberos database
Aug 11 18:58:20 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.129@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:58:20 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.129@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:58:20.557369 13430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:20.547763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:44265 (local address 127.29.112.129:37311)
0811 18:58:20.548133 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:20.548155 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:20.548276 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:20.548680 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:20.548684 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:20.549014 (+ 330us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:58:20.549150 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:20.549157 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:20.550860 (+ 1703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:20.550867 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:20.552138 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:20.552154 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:20.553194 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:20.553198 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:20.553342 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:20.553995 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:58:20.554013 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:58:20.557013 (+ 3000us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.190:44265: Server oryx/127.29.112.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":88}
W20260811 18:58:20.558176 13426 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:58:20.548135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.129:37311 (local address 127.29.112.190:44265)
0811 18:58:20.548347 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:20.548356 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:20.548386 (+ 30us) server_negotiation.cc:407] Connection header received
0811 18:58:20.548445 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:20.548453 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:20.548536 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:20.548642 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:20.549305 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:20.550601 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:20.552303 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:20.553026 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:20.553199 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:20.557704 (+ 4505us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.129:37311: BlockingRecv error: recv got EOF from 127.29.112.129:37311 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 11 18:58:20 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.129@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:58:20 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.129@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:58:20.566988 13430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:20.558132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:44265 (local address 127.29.112.129:56491)
0811 18:58:20.558378 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:20.558400 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:20.558549 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:20.559398 (+ 849us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:20.559406 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:20.559752 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:58:20.559896 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:20.559904 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:20.561019 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:20.561031 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:20.561927 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:20.561938 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:20.562669 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:20.562674 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:20.562849 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:20.563532 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:58:20.563548 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:58:20.566689 (+ 3141us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.190:44265: Server oryx/127.29.112.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144}
W20260811 18:58:20.567452 13432 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:58:20.558224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.129:56491 (local address 127.29.112.190:44265)
0811 18:58:20.559098 (+ 874us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:20.559106 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:20.559129 (+ 23us) server_negotiation.cc:407] Connection header received
0811 18:58:20.559183 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:20.559187 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:20.559242 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:20.559360 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:20.560077 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:20.560865 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:20.562096 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:20.562502 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:20.562660 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:20.566949 (+ 4289us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.129:56491: BlockingRecv error: recv got EOF from 127.29.112.129:56491 (error 108)
Metrics: {"server-negotiator.queue_time_us":739,"thread_start_us":617,"threads_started":1}
W20260811 18:58:20.567539 13419 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 18:58:20.876817 13431 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:58:20.877148 13431 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:58:20.877311 13431 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:58:20.884629 13431 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:58:20.884732 13431 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:58:20.885004 13431 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.130
I20260811 18:58:20.894322 13431 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:44253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/krb5kdc/oryx_127.29.112.130.keytab
--principal=oryx/127.29.112.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.130:43559
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.112.130
--webserver_port=36381
--tserver_master_addrs=127.29.112.190:44265
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.130
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:58:20.895540 13431 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:58:20.896767 13431 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:58:20.911756 13441 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:20.911790 13442 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:20.911720 13447 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:58:20.912777 13431 server_base.cc:1034] running on GCE node
I20260811 18:58:20.913380 13431 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:58:20.914284 13431 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:58:20.915518 13431 hybrid_clock.cc:648] HybridClock initialized: now 1786474700915459 us; error 68 us; skew 500 ppm
I20260811 18:58:20.915863 13431 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:58:20 dist-test-slave-h6n0 krb5kdc[12231](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474700, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.112.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:58:20.919116 13431 init.cc:377] Logged in from keytab as oryx/127.29.112.130@KRBTEST.COM (short username oryx)
I20260811 18:58:20.920440 13431 webserver.cc:460] Webserver started at http://127.29.112.130:36381/ using document root <none> and password file <none>
I20260811 18:58:20.921056 13431 fs_manager.cc:359] Metadata directory not provided
I20260811 18:58:20.921234 13431 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:58:20.926801 13431 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 18:58:20.929360 13457 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:20.930750 13431 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:58:20.930929 13431 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/ts-1/wal
uuid: "2d6953d91b2b4b77b994b7ff6e421ae7"
format_stamp: "Formatted at 2026-08-11 18:58:18 on dist-test-slave-h6n0"
server_key: "c26c8d9e18a1580da924ef03e2c75a4c"
server_key_iv: "1228eee75b189f90f80ac127865e38d6"
server_key_version: "encryptionkey@0"
I20260811 18:58:20.931980 13431 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:58:20.952040 13431 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:58:20.956341 13431 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:58:20.956640 13431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:58:20.957707 13431 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:58:20.959100 13431 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:58:20.959231 13431 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:20.959364 13431 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:58:20.959465 13431 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:21.000084 13431 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.130:43559
I20260811 18:58:21.000139 13638 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.130:43559 every 8 connection(s)
I20260811 18:58:21.002380 13431 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/ts-1/data/info.pb
I20260811 18:58:21.003324 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 13431
I20260811 18:58:21.004303 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.131:39027
--local_ip_for_outbound_sockets=127.29.112.131
--tserver_master_addrs=127.29.112.190:44265
--webserver_port=40177
--webserver_interface=127.29.112.131
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.131.keytab
--principal=kudu/127.29.112.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:44253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.29.112.131
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/krb5kdc/oryx_127.29.112.131.keytab with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 18:58:21 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.130@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:58:21 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.130@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
W20260811 18:58:21.026300 13432 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:58:21.006647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.130:46021 (local address 127.29.112.190:44265)
0811 18:58:21.006902 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:21.006908 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:21.008978 (+ 2070us) server_negotiation.cc:407] Connection header received
0811 18:58:21.011794 (+ 2816us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:21.011803 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:21.011907 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:21.012040 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:21.014066 (+ 2026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:21.015741 (+ 1675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:21.018177 (+ 2436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:21.019122 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:21.019320 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:21.026011 (+ 6691us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.130:46021: BlockingRecv error: recv got EOF from 127.29.112.130:46021 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
I20260811 18:58:21.026764 13645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:21.007547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:44265 (local address 127.29.112.130:46021)
0811 18:58:21.008771 (+ 1224us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:21.008826 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:21.011459 (+ 2633us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:21.012329 (+ 870us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:21.012341 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:21.013115 (+ 774us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:58:21.013864 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:21.013886 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:21.016009 (+ 2123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:21.016015 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:21.017943 (+ 1928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:21.017959 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:21.019415 (+ 1456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:21.019423 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:21.019703 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:21.020678 (+ 975us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:58:21.020711 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:58:21.025687 (+ 4976us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.190:44265: Server oryx/127.29.112.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":868,"thread_start_us":570,"threads_started":1}
W20260811 18:58:21.027463 13639 heartbeater.cc:644] Failed to heartbeat to 127.29.112.190:44265 (0 consecutive failures): Not authorized: Failed to ping master at 127.29.112.190:44265: Client connection negotiation failed: client connection to 127.29.112.190:44265: Server oryx/127.29.112.190@KRBTEST.COM not found in Kerberos database
Aug 11 18:58:21 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.130@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:58:21 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.130@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:58:21.039733 13645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:21.028071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:44265 (local address 127.29.112.130:35077)
0811 18:58:21.028307 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:21.028328 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:21.028436 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:21.028760 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:21.028767 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:21.029268 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:58:21.029480 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:21.029494 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:21.031038 (+ 1544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:21.031047 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:21.032749 (+ 1702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:21.032768 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:21.033667 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:21.033675 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:21.033924 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:21.034636 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:58:21.034666 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:58:21.039362 (+ 4696us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.190:44265: Server oryx/127.29.112.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":83}
W20260811 18:58:21.040381 13432 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:58:21.028236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.130:35077 (local address 127.29.112.190:44265)
0811 18:58:21.028399 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:21.028403 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:21.028420 (+ 17us) server_negotiation.cc:407] Connection header received
0811 18:58:21.028546 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:21.028549 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:21.028614 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:21.028695 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:21.029661 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:21.030879 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:21.033037 (+ 2158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:21.033486 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:21.033600 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:21.039661 (+ 6061us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.130:35077: BlockingRecv error: recv got EOF from 127.29.112.130:35077 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 11 18:58:21 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.130@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:58:21 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.130@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:58:21.050428 13645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:21.040633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:44265 (local address 127.29.112.130:33255)
0811 18:58:21.040827 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:21.040849 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:21.040966 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:21.041496 (+ 530us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:21.041502 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:21.041886 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:58:21.042068 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:21.042077 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:21.043282 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:21.043287 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:21.044423 (+ 1136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:21.044438 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:21.045335 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:21.045341 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:21.045557 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:21.046276 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:58:21.046295 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:58:21.050080 (+ 3785us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.190:44265: Server oryx/127.29.112.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":69}
W20260811 18:58:21.050850 13639 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 18:58:21.051185 13432 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:58:21.040998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.130:33255 (local address 127.29.112.190:44265)
0811 18:58:21.041191 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:21.041195 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:21.041210 (+ 15us) server_negotiation.cc:407] Connection header received
0811 18:58:21.041263 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:21.041266 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:21.041342 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:21.041462 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:21.042265 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:21.043123 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:21.044606 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:21.045167 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:21.045310 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:21.050472 (+ 5162us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.130:33255: BlockingRecv error: recv got EOF from 127.29.112.130:33255 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260811 18:58:21.377831 13646 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:58:21.378166 13646 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:58:21.378345 13646 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:58:21.385797 13646 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:58:21.385946 13646 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:58:21.386119 13646 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.131
I20260811 18:58:21.395666 13646 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:44253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/krb5kdc/oryx_127.29.112.131.keytab
--principal=oryx/127.29.112.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.131:39027
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.112.131
--webserver_port=40177
--tserver_master_addrs=127.29.112.190:44265
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.131
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:58:21.396886 13646 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:58:21.398157 13646 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:58:21.411350 13654 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:21.411309 13656 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:21.411240 13661 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:58:21.413269 13646 server_base.cc:1034] running on GCE node
I20260811 18:58:21.413921 13646 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:58:21.414917 13646 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:58:21.416147 13646 hybrid_clock.cc:648] HybridClock initialized: now 1786474701416105 us; error 58 us; skew 500 ppm
I20260811 18:58:21.416450 13646 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:58:21 dist-test-slave-h6n0 krb5kdc[12231](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474701, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.112.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:58:21.419613 13646 init.cc:377] Logged in from keytab as oryx/127.29.112.131@KRBTEST.COM (short username oryx)
I20260811 18:58:21.421149 13646 webserver.cc:460] Webserver started at http://127.29.112.131:40177/ using document root <none> and password file <none>
I20260811 18:58:21.421881 13646 fs_manager.cc:359] Metadata directory not provided
I20260811 18:58:21.422052 13646 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:58:21.427980 13646 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.004s
I20260811 18:58:21.431286 13676 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:21.432665 13646 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:58:21.432859 13646 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/ts-2/wal
uuid: "1988db1b14a242b5b586b76f340bcdfb"
format_stamp: "Formatted at 2026-08-11 18:58:18 on dist-test-slave-h6n0"
server_key: "44fdf3772b9574957929b4018ef77a6d"
server_key_iv: "ad13cb19c4ce5c6b378a07d2c36cb18a"
server_key_version: "encryptionkey@0"
I20260811 18:58:21.434012 13646 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:58:21.458618 13646 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:58:21.463150 13646 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:58:21.463421 13646 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:58:21.464437 13646 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:58:21.466055 13646 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:58:21.466375 13646 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:21.466586 13646 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:58:21.466777 13646 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260811 18:58:21.505373 13646 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.131:39027
I20260811 18:58:21.505407 13862 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.131:39027 every 8 connection(s)
I20260811 18:58:21.507114 13646 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474615205622-30146-0/minicluster-data/ts-2/data/info.pb
I20260811 18:58:21.512295 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 13646
Aug 11 18:58:21 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.131@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:58:21 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.131@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
W20260811 18:58:21.528599 13432 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:58:21.510394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.131:48039 (local address 127.29.112.190:44265)
0811 18:58:21.510645 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:21.510650 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:21.511688 (+ 1038us) server_negotiation.cc:407] Connection header received
0811 18:58:21.514384 (+ 2696us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:21.514391 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:21.514460 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:21.514575 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:21.516606 (+ 2031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:21.517945 (+ 1339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:21.520570 (+ 2625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:21.521560 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:21.521805 (+ 245us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:21.528301 (+ 6496us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.131:48039: BlockingRecv error: recv got EOF from 127.29.112.131:48039 (error 108)
Metrics: {"mutex_wait_us":104,"server-negotiator.queue_time_us":61}
I20260811 18:58:21.529290 13873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:21.510757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:44265 (local address 127.29.112.131:48039)
0811 18:58:21.511494 (+ 737us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:21.511531 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:21.514123 (+ 2592us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:21.514855 (+ 732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:21.514868 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:21.515629 (+ 761us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:58:21.516449 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:21.516476 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:21.518332 (+ 1856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:21.518338 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:21.520388 (+ 2050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:21.520401 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:21.522446 (+ 2045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:21.522462 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:21.522764 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:21.523398 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:58:21.523419 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:58:21.528032 (+ 4613us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.190:44265: Server oryx/127.29.112.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":303,"threads_started":1}
W20260811 18:58:21.530309 13869 heartbeater.cc:644] Failed to heartbeat to 127.29.112.190:44265 (0 consecutive failures): Not authorized: Failed to ping master at 127.29.112.190:44265: Client connection negotiation failed: client connection to 127.29.112.190:44265: Server oryx/127.29.112.190@KRBTEST.COM not found in Kerberos database
I20260811 18:58:21.532975 13875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:21.513424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52886 (local address 127.29.112.190:44265)
0811 18:58:21.514165 (+ 741us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:21.514169 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:21.514184 (+ 15us) server_negotiation.cc:407] Connection header received
0811 18:58:21.514237 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:21.514241 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:21.514306 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:21.514448 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:21.516606 (+ 2158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:21.517945 (+ 1339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:21.520245 (+ 2300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:21.521258 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:21.521450 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:21.524979 (+ 3529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:58:21.524988 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:58:21.525000 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:58:21.525068 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:58:21.528868 (+ 3800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:21.530102 (+ 1234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:21.530108 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:21.530114 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:21.530269 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:21.530788 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:21.530792 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:21.530794 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:21.531446 (+ 652us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:58:21.531788 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:21.532209 (+ 421us) server_negotiation.cc:299] Negotiation successful
0811 18:58:21.532515 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":121,"server-negotiator.queue_time_us":661,"thread_start_us":548,"threads_started":1}
Aug 11 18:58:21 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.131@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:58:21 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.131@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:58:21.539846 13873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:21.531149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:44265 (local address 127.29.112.131:36145)
0811 18:58:21.531550 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:21.531570 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:21.531685 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:21.532052 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:21.532057 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:21.532457 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:58:21.532625 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:21.532634 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:21.533954 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:21.533960 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:21.535144 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:21.535164 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:21.536006 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:21.536012 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:21.536196 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:21.536793 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:58:21.536817 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:58:21.539659 (+ 2842us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.190:44265: Server oryx/127.29.112.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":272}
W20260811 18:58:21.540118 13432 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:58:21.531366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.131:36145 (local address 127.29.112.190:44265)
0811 18:58:21.531547 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:21.531553 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:21.531644 (+ 91us) server_negotiation.cc:407] Connection header received
0811 18:58:21.531805 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:21.531812 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:21.531885 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:21.531993 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:21.532769 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:21.533797 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:21.535338 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:21.535811 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:21.535955 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:21.539843 (+ 3888us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.131:36145: BlockingRecv error: recv got EOF from 127.29.112.131:36145 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 11 18:58:21 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.131@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:58:21 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.131@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:58:21.547652 13873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:21.540495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:44265 (local address 127.29.112.131:40615)
0811 18:58:21.540681 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:21.540696 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:21.540768 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:21.541024 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:21.541027 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:21.541284 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:58:21.541417 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:21.541424 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:21.542363 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:21.542372 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:21.543348 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:21.543360 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:21.543991 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:21.543995 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:21.544112 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:21.544584 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:58:21.544599 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:58:21.547460 (+ 2861us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.190:44265: Server oryx/127.29.112.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":72}
W20260811 18:58:21.548019 13869 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 18:58:21.548396 13432 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:58:21.540595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.131:40615 (local address 127.29.112.190:44265)
0811 18:58:21.540795 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:21.540799 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:21.540815 (+ 16us) server_negotiation.cc:407] Connection header received
0811 18:58:21.540850 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:21.540854 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:21.540901 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:21.540985 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:21.541541 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:21.542213 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:21.543467 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:21.543855 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:21.543942 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:21.547868 (+ 3926us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.131:40615: BlockingRecv error: recv got EOF from 127.29.112.131:40615 (error 108)
Metrics: {"server-negotiator.queue_time_us":106}
Aug 11 18:58:21 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.129@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:58:21 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.129@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:58:21.577157 13876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:21.568117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:44265 (local address 127.29.112.129:40221)
0811 18:58:21.568674 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:21.568690 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:21.568850 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:21.569223 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:21.569228 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:21.569713 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:58:21.569911 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:21.569920 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:21.570980 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:21.570992 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:21.572092 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:21.572105 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:21.573081 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:21.573090 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:21.573257 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:21.574059 (+ 802us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:58:21.574077 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:58:21.576878 (+ 2801us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.190:44265: Server oryx/127.29.112.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":308,"threads_started":1}
W20260811 18:58:21.577310 13432 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:58:21.568228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.129:40221 (local address 127.29.112.190:44265)
0811 18:58:21.568472 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:21.568479 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:21.568777 (+ 298us) server_negotiation.cc:407] Connection header received
0811 18:58:21.568990 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:21.568996 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:21.569066 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:21.569170 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:21.570058 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:21.570835 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:21.572231 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:21.572843 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:21.572995 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:21.577109 (+ 4114us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.129:40221: BlockingRecv error: recv got EOF from 127.29.112.129:40221 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
Aug 11 18:58:22 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.130@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:58:22 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.130@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:58:22.061297 13877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:22.051686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:44265 (local address 127.29.112.130:50217)
0811 18:58:22.052303 (+ 617us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:22.052323 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:22.052455 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:22.052820 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:22.052824 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:22.053325 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:58:22.053534 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:22.053544 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:22.054662 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:22.054667 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:22.055654 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:22.055666 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:22.056414 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:22.056419 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:22.056578 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:22.057298 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:58:22.057316 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:58:22.060984 (+ 3668us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.190:44265: Server oryx/127.29.112.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":316,"threads_started":1}
W20260811 18:58:22.061530 13432 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:58:22.051675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.130:50217 (local address 127.29.112.190:44265)
0811 18:58:22.051938 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:22.051948 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:22.052461 (+ 513us) server_negotiation.cc:407] Connection header received
0811 18:58:22.052547 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:22.052551 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:22.052633 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:22.052722 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:22.053694 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:22.054414 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:22.055819 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:22.056227 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:22.056328 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:22.061242 (+ 4914us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.130:50217: BlockingRecv error: recv got EOF from 127.29.112.130:50217 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 11 18:58:22 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.131@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:58:22 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.131@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:58:22.557992 13880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:22.548703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:44265 (local address 127.29.112.131:44273)
0811 18:58:22.549270 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:22.549286 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:22.549408 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:22.549703 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:22.549706 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:22.550156 (+ 450us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:58:22.550347 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:22.550367 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:22.551965 (+ 1598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:22.551977 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:22.553083 (+ 1106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:22.553096 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:22.553848 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:22.553854 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:22.554026 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:22.554730 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:58:22.554747 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:58:22.557687 (+ 2940us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.190:44265: Server oryx/127.29.112.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":312,"threads_started":1}
W20260811 18:58:22.558051 13432 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:58:22.548784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.131:44273 (local address 127.29.112.190:44265)
0811 18:58:22.548966 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:22.548971 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:22.549404 (+ 433us) server_negotiation.cc:407] Connection header received
0811 18:58:22.549497 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:22.549501 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:22.549567 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:22.549651 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:22.550737 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:22.551810 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:22.553231 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:22.553655 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:22.553802 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:22.557870 (+ 4068us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.131:44273: BlockingRecv error: recv got EOF from 127.29.112.131:44273 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 11 18:58:22 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.129@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:58:22 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.129@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:58:22.586993 13882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:22.578165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:44265 (local address 127.29.112.129:57817)
0811 18:58:22.578755 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:22.578771 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:22.578888 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:22.579578 (+ 690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:22.579582 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:22.580020 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:58:22.580195 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:22.580203 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:22.581544 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:22.581549 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:22.582449 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:22.582460 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:22.583224 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:22.583228 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:22.583340 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:22.583953 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:58:22.583974 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:58:22.586750 (+ 2776us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.190:44265: Server oryx/127.29.112.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":307,"threads_started":1}
W20260811 18:58:22.587205 13432 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:58:22.578266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.129:57817 (local address 127.29.112.190:44265)
0811 18:58:22.578478 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:22.578482 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:22.578867 (+ 385us) server_negotiation.cc:407] Connection header received
0811 18:58:22.579182 (+ 315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:22.579188 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:22.579395 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:22.579507 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:22.580372 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:22.581371 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:22.582653 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:22.583081 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:22.583175 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:22.586950 (+ 3775us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.129:57817: BlockingRecv error: recv got EOF from 127.29.112.129:57817 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
Aug 11 18:58:23 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.130@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:58:23 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.130@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:58:23.072024 13883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:23.062922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:44265 (local address 127.29.112.130:55125)
0811 18:58:23.063529 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:23.063554 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:23.063710 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:23.064290 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:23.064298 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:23.064992 (+ 694us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:58:23.065274 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:23.065288 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:23.066305 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:23.066309 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:23.067312 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:23.067324 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:23.068114 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:23.068119 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:23.068235 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:23.068891 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:58:23.068909 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:58:23.071754 (+ 2845us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.190:44265: Server oryx/127.29.112.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":525,"threads_started":1}
W20260811 18:58:23.072381 13432 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:58:23.063241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.130:55125 (local address 127.29.112.190:44265)
0811 18:58:23.063452 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:23.063462 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:23.063655 (+ 193us) server_negotiation.cc:407] Connection header received
0811 18:58:23.063917 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:23.063925 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:23.064050 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:23.064153 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:23.065444 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:23.066158 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:23.067442 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:23.067925 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:23.068029 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:23.071998 (+ 3969us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.130:55125: BlockingRecv error: recv got EOF from 127.29.112.130:55125 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 11 18:58:23 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.131@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:58:23 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.131@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:58:23.567732 13884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:23.558973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:44265 (local address 127.29.112.131:54697)
0811 18:58:23.559542 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:23.559559 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:23.559671 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:23.559959 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:23.559962 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:23.560385 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:58:23.560557 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:23.560565 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:23.561563 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:23.561566 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:23.562608 (+ 1042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:23.562631 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:23.563593 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:23.563599 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:23.563776 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:23.564498 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:58:23.564515 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:58:23.567377 (+ 2862us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.190:44265: Server oryx/127.29.112.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":329,"threads_started":1}
W20260811 18:58:23.567842 13432 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:58:23.559132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.131:54697 (local address 127.29.112.190:44265)
0811 18:58:23.559316 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:23.559321 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:23.559635 (+ 314us) server_negotiation.cc:407] Connection header received
0811 18:58:23.559768 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:23.559772 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:23.559833 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:23.559915 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:23.560690 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:23.561426 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:23.562792 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:23.563432 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:23.563571 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:23.567594 (+ 4023us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.131:54697: BlockingRecv error: recv got EOF from 127.29.112.131:54697 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 11 18:58:23 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.129@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:58:23 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.129@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:58:23.596903 13886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:23.587952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:44265 (local address 127.29.112.129:52173)
0811 18:58:23.588458 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:23.588482 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:23.588622 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:23.589089 (+ 467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:23.589093 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:23.589502 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:58:23.589715 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:23.589726 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:23.590808 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:23.590813 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:23.591703 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:23.591713 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:23.592431 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:23.592436 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:23.592611 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:23.593334 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:58:23.593352 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:58:23.596638 (+ 3286us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.190:44265: Server oryx/127.29.112.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":250,"threads_started":1}
W20260811 18:58:23.597055 13432 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:58:23.588553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.129:52173 (local address 127.29.112.190:44265)
0811 18:58:23.588772 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:23.588776 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:23.588790 (+ 14us) server_negotiation.cc:407] Connection header received
0811 18:58:23.588857 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:23.588862 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:23.588943 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:23.589044 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:23.589867 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:23.590666 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:23.591851 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:23.592274 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:23.592404 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:23.596825 (+ 4421us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.129:52173: BlockingRecv error: recv got EOF from 127.29.112.129:52173 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
Aug 11 18:58:24 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.130@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:58:24 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.130@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:58:24.081770 13888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:24.073454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:44265 (local address 127.29.112.130:47163)
0811 18:58:24.073986 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:24.074008 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:24.074142 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:24.074495 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:24.074499 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:24.074934 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:58:24.075101 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:24.075110 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:24.076136 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:24.076142 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:24.077081 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:24.077093 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:24.077800 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:24.077805 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:24.077988 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:24.078651 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:58:24.078668 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:58:24.081514 (+ 2846us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.190:44265: Server oryx/127.29.112.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":289,"threads_started":1}
W20260811 18:58:24.081980 13432 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:58:24.073911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.130:47163 (local address 127.29.112.190:44265)
0811 18:58:24.074116 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:24.074122 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:24.074143 (+ 21us) server_negotiation.cc:407] Connection header received
0811 18:58:24.074262 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:24.074267 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:24.074352 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:24.074465 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:24.075259 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:24.075985 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:24.077228 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:24.077647 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:24.077775 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:24.081707 (+ 3932us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.130:47163: BlockingRecv error: recv got EOF from 127.29.112.130:47163 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 11 18:58:24 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.131@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:58:24 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.131@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:58:24.577215 13890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:24.568722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:44265 (local address 127.29.112.131:32803)
0811 18:58:24.569272 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:24.569289 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:24.569404 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:24.569709 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:24.569712 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:24.570057 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:58:24.570234 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:24.570242 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:24.571326 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:24.571330 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:24.572362 (+ 1032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:24.572374 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:24.573102 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:24.573107 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:24.573279 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:24.573898 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:58:24.573916 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:58:24.576901 (+ 2985us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.190:44265: Server oryx/127.29.112.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":303,"threads_started":1}
W20260811 18:58:24.577467 13432 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:58:24.568852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.131:32803 (local address 127.29.112.190:44265)
0811 18:58:24.569045 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:24.569050 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:24.569368 (+ 318us) server_negotiation.cc:407] Connection header received
0811 18:58:24.569506 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:24.569510 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:24.569573 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:24.569667 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:24.570394 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:24.571173 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:24.572550 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:24.572939 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:24.573076 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:24.577210 (+ 4134us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.131:32803: BlockingRecv error: recv got EOF from 127.29.112.131:32803 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 11 18:58:24 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.129@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:58:24 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.129@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:58:24.607301 13893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:24.597839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:44265 (local address 127.29.112.129:42685)
0811 18:58:24.598341 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:24.598357 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:24.598577 (+ 220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:24.599294 (+ 717us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:24.599306 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:24.599786 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:58:24.599946 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:24.599956 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:24.601228 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:24.601234 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:24.602245 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:24.602258 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:24.603058 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:24.603064 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:24.603257 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:24.603970 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:58:24.603988 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:58:24.606992 (+ 3004us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.190:44265: Server oryx/127.29.112.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":296,"threads_started":1}
W20260811 18:58:24.607519 13432 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:58:24.597877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.129:42685 (local address 127.29.112.190:44265)
0811 18:58:24.598097 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:24.598106 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:24.598454 (+ 348us) server_negotiation.cc:407] Connection header received
0811 18:58:24.598710 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:24.598715 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:24.598785 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:24.599177 (+ 392us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:24.600109 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:24.600889 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:24.602415 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:24.602905 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:24.603039 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:24.607285 (+ 4246us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.129:42685: BlockingRecv error: recv got EOF from 127.29.112.129:42685 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 11 18:58:25 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.130@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:58:25 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.130@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:58:25.096926 13894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:25.082823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:44265 (local address 127.29.112.130:47049)
0811 18:58:25.083352 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:25.083369 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:25.083539 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:25.084010 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:25.084015 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:25.084557 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:58:25.084760 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:25.084772 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:25.086161 (+ 1389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:25.086164 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:25.087200 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:25.087212 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:25.087861 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:25.087866 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:25.088001 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:25.093281 (+ 5280us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:58:25.093299 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:58:25.096582 (+ 3283us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.190:44265: Server oryx/127.29.112.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":283,"threads_started":1}
W20260811 18:58:25.097049 13432 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:58:25.082911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.130:47049 (local address 127.29.112.190:44265)
0811 18:58:25.083172 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:25.083179 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:25.083461 (+ 282us) server_negotiation.cc:407] Connection header received
0811 18:58:25.083688 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:25.083693 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:25.083795 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:25.083910 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:25.085025 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:25.086038 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:25.087318 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:25.087710 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:25.087805 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:25.096776 (+ 8971us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.130:47049: BlockingRecv error: recv got EOF from 127.29.112.130:47049 (error 108)
Metrics: {"server-negotiator.queue_time_us":94}
Aug 11 18:58:25 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.131@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:58:25 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.131@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:58:25.587641 13896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:25.578324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:44265 (local address 127.29.112.131:54501)
0811 18:58:25.578940 (+ 616us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:25.578959 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:25.579092 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:25.579410 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:25.579414 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:25.579816 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:58:25.579986 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:25.579995 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:25.581200 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:25.581204 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:25.582246 (+ 1042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:25.582259 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:25.583043 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:25.583047 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:25.583181 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:25.583968 (+ 787us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:58:25.584000 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:58:25.587405 (+ 3405us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.190:44265: Server oryx/127.29.112.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":394,"threads_started":1}
W20260811 18:58:25.587939 13432 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:58:25.578378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.131:54501 (local address 127.29.112.190:44265)
0811 18:58:25.578678 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:25.578689 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:25.579063 (+ 374us) server_negotiation.cc:407] Connection header received
0811 18:58:25.579199 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:25.579203 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:25.579272 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:25.579362 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:25.580140 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:25.580904 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:25.582428 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:25.582861 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:25.582969 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:25.587657 (+ 4688us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.131:54501: BlockingRecv error: recv got EOF from 127.29.112.131:54501 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 11 18:58:25 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.129@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:58:25 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.129@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:58:25.617949 13900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:25.608465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:44265 (local address 127.29.112.129:59589)
0811 18:58:25.609091 (+ 626us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:25.609116 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:25.609269 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:25.609664 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:25.609668 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:25.610233 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:58:25.610412 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:25.610422 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:25.611572 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:25.611579 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:25.612497 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:25.612508 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:25.613225 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:25.613231 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:25.613408 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:25.614107 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:58:25.614125 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:58:25.617562 (+ 3437us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.190:44265: Server oryx/127.29.112.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":299,"threads_started":1}
W20260811 18:58:25.618044 13432 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:58:25.608528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.129:59589 (local address 127.29.112.190:44265)
0811 18:58:25.608808 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:25.608814 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:25.609215 (+ 401us) server_negotiation.cc:407] Connection header received
0811 18:58:25.609432 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:25.609437 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:25.609518 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:25.609626 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:25.610620 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:25.611432 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:25.612644 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:25.613069 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:25.613209 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:25.617780 (+ 4571us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.129:59589: BlockingRecv error: recv got EOF from 127.29.112.129:59589 (error 108)
Metrics: {"server-negotiator.queue_time_us":109}
Aug 11 18:58:26 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.130@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:58:26 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.130@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:58:26.109561 13901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:26.098012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:44265 (local address 127.29.112.130:54455)
0811 18:58:26.098685 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:26.098710 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:26.098859 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:26.099288 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:26.099295 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:26.099962 (+ 667us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:58:26.100263 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:26.100279 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:26.101351 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:26.101359 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:26.103318 (+ 1959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:26.103337 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:26.104167 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:26.104176 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:26.104408 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:26.105293 (+ 885us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:58:26.105324 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:58:26.109234 (+ 3910us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.190:44265: Server oryx/127.29.112.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":319,"threads_started":1}
W20260811 18:58:26.109838 13432 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:58:26.098309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.130:54455 (local address 127.29.112.190:44265)
0811 18:58:26.098484 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:26.098491 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:26.098800 (+ 309us) server_negotiation.cc:407] Connection header received
0811 18:58:26.099037 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:26.099042 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:26.099133 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:26.099223 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:26.100447 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:26.101181 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:26.103492 (+ 2311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:26.104000 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:26.104124 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:26.109506 (+ 5382us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.130:54455: BlockingRecv error: recv got EOF from 127.29.112.130:54455 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 11 18:58:26 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.131@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:58:26 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.131@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:58:26.597384 13902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:26.588723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:44265 (local address 127.29.112.131:38307)
0811 18:58:26.589308 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:26.589324 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:26.589496 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:26.589844 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:26.589848 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:26.590299 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:58:26.590460 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:26.590468 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:26.591575 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:26.591583 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:26.592499 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:26.592510 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:26.593360 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:26.593366 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:26.593550 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:26.594231 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:58:26.594248 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:58:26.597135 (+ 2887us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.190:44265: Server oryx/127.29.112.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":311,"threads_started":1}
W20260811 18:58:26.597509 13432 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:58:26.588859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.131:38307 (local address 127.29.112.190:44265)
0811 18:58:26.589128 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:26.589136 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:26.589430 (+ 294us) server_negotiation.cc:407] Connection header received
0811 18:58:26.589625 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:26.589631 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:26.589705 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:26.589812 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:26.590668 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:26.591439 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:26.592646 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:26.593194 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:26.593347 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:26.597336 (+ 3989us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.131:38307: BlockingRecv error: recv got EOF from 127.29.112.131:38307 (error 108)
Metrics: {"server-negotiator.queue_time_us":111}
Aug 11 18:58:26 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.129@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:58:26 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.129@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:58:26.627075 13903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:26.618996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:44265 (local address 127.29.112.129:47873)
0811 18:58:26.619461 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:26.619477 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:26.619639 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:26.619987 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:26.619992 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:26.620412 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:58:26.620594 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:26.620602 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:26.621586 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:26.621592 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:26.622501 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:26.622518 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:26.623290 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:26.623296 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:26.623478 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:26.624072 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:58:26.624089 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:58:26.626850 (+ 2761us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.190:44265: Server oryx/127.29.112.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":252,"threads_started":1}
W20260811 18:58:26.627400 13432 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:58:26.619001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.129:47873 (local address 127.29.112.190:44265)
0811 18:58:26.619202 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:26.619209 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:26.619577 (+ 368us) server_negotiation.cc:407] Connection header received
0811 18:58:26.619771 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:26.619777 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:26.619851 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:26.619952 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:26.620741 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:26.621433 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:26.622678 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:26.623134 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:26.623266 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:26.627041 (+ 3775us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.129:47873: BlockingRecv error: recv got EOF from 127.29.112.129:47873 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 11 18:58:27 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.130@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:58:27 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.130@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:58:27.120074 13904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:27.110577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:44265 (local address 127.29.112.130:41659)
0811 18:58:27.111110 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:27.111125 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:27.111289 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:27.111661 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:27.111666 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:27.112123 (+ 457us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:58:27.112280 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:27.112289 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:27.113415 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:27.113424 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:27.114375 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:27.114386 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:27.115939 (+ 1553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:27.115949 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:27.116262 (+ 313us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:27.117059 (+ 797us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:58:27.117076 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:58:27.119827 (+ 2751us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.190:44265: Server oryx/127.29.112.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":294,"threads_started":1}
W20260811 18:58:27.120357 13432 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:58:27.110667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.130:41659 (local address 127.29.112.190:44265)
0811 18:58:27.110876 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:27.110885 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:27.111221 (+ 336us) server_negotiation.cc:407] Connection header received
0811 18:58:27.111446 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:27.111450 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:27.111524 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:27.111630 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:27.112438 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:27.113259 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:27.114549 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:27.115494 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:27.115738 (+ 244us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:27.120094 (+ 4356us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.130:41659: BlockingRecv error: recv got EOF from 127.29.112.130:41659 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 11 18:58:27 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.131@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:58:27 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.131@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:58:27.606609 13905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:27.598620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:44265 (local address 127.29.112.131:45235)
0811 18:58:27.599151 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:27.599167 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:27.599283 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:27.599591 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:27.599594 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:27.599969 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:58:27.600129 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:27.600136 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:27.601117 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:27.601122 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:27.602019 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:27.602030 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:27.602718 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:27.602722 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:27.602848 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:27.603428 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:58:27.603444 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:58:27.606270 (+ 2826us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.190:44265: Server oryx/127.29.112.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":314,"threads_started":1}
W20260811 18:58:27.606751 13432 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:58:27.598638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.131:45235 (local address 127.29.112.190:44265)
0811 18:58:27.598858 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:27.598868 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:27.599254 (+ 386us) server_negotiation.cc:407] Connection header received
0811 18:58:27.599399 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:27.599403 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:27.599467 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:27.599547 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:27.600248 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:27.600978 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:27.602145 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:27.602584 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:27.602674 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:27.606475 (+ 3801us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.131:45235: BlockingRecv error: recv got EOF from 127.29.112.131:45235 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 11 18:58:27 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.129@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:58:27 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.129@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:58:27.639654 13906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:27.628404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:44265 (local address 127.29.112.129:40363)
0811 18:58:27.629165 (+ 761us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:27.629188 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:27.629339 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:27.629804 (+ 465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:27.629808 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:27.630326 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:58:27.630622 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:27.630635 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:27.631725 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:27.631731 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:27.632728 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:27.632754 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:27.633752 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:27.633766 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:27.634103 (+ 337us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:27.634869 (+ 766us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:58:27.634888 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:58:27.639375 (+ 4487us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.190:44265: Server oryx/127.29.112.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":609,"thread_start_us":374,"threads_started":1}
W20260811 18:58:27.639818 13432 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:58:27.628504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.129:40363 (local address 127.29.112.190:44265)
0811 18:58:27.628729 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:27.628734 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:27.629293 (+ 559us) server_negotiation.cc:407] Connection header received
0811 18:58:27.629496 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:27.629503 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:27.629623 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:27.629735 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:27.630772 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:27.631505 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:27.632926 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:27.633382 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:27.633486 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:27.639574 (+ 6088us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.129:40363: BlockingRecv error: recv got EOF from 127.29.112.129:40363 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 11 18:58:28 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.130@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:58:28 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.130@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:58:28.129657 13909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:28.120992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:44265 (local address 127.29.112.130:60525)
0811 18:58:28.121533 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:28.121551 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:28.121672 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:28.122001 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:28.122005 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:28.122471 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:58:28.122741 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:28.122750 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:28.123685 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:28.123688 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:28.124630 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:28.124641 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:28.125360 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:28.125365 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:28.125495 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:28.126116 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:58:28.126134 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:58:28.129399 (+ 3265us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.190:44265: Server oryx/127.29.112.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":250,"threads_started":1}
W20260811 18:58:28.129874 13432 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:58:28.121053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.130:60525 (local address 127.29.112.190:44265)
0811 18:58:28.121325 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:28.121334 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:28.121635 (+ 301us) server_negotiation.cc:407] Connection header received
0811 18:58:28.121772 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:28.121776 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:28.121847 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:28.121925 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:28.122870 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:28.123561 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:28.124753 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:28.125184 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:28.125279 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:28.129612 (+ 4333us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.130:60525: BlockingRecv error: recv got EOF from 127.29.112.130:60525 (error 108)
Metrics: {"server-negotiator.queue_time_us":113}
Aug 11 18:58:28 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.131@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:58:28 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.131@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:58:28.616593 13910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:28.607598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:44265 (local address 127.29.112.131:36043)
0811 18:58:28.608192 (+ 594us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:28.608213 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:28.608432 (+ 219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:28.608785 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:28.608790 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:28.609235 (+ 445us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:58:28.609414 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:28.609424 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:28.610454 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:28.610460 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:28.611411 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:28.611423 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:28.612132 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:28.612138 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:28.612311 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:28.612964 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:58:28.612982 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:58:28.616227 (+ 3245us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.190:44265: Server oryx/127.29.112.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":247,"threads_started":1}
W20260811 18:58:28.616698 13432 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:58:28.607623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.131:36043 (local address 127.29.112.190:44265)
0811 18:58:28.607799 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:28.607804 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:28.608304 (+ 500us) server_negotiation.cc:407] Connection header received
0811 18:58:28.608568 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:28.608573 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:28.608645 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:28.608749 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:28.609564 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:28.610303 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:28.611565 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:28.611975 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:28.612109 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:28.616461 (+ 4352us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.131:36043: BlockingRecv error: recv got EOF from 127.29.112.131:36043 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 11 18:58:28 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.129@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:58:28 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.129@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:58:28.650722 13913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:28.640821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:44265 (local address 127.29.112.129:45423)
0811 18:58:28.641701 (+ 880us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:28.641718 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:28.641860 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:28.642275 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:28.642278 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:28.642864 (+ 586us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:58:28.643165 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:28.643179 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:28.644302 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:28.644306 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:28.645229 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:28.645239 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:28.646059 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:28.646063 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:28.646179 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:28.647323 (+ 1144us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:58:28.647350 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:58:28.650397 (+ 3047us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.190:44265: Server oryx/127.29.112.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":730,"thread_start_us":606,"threads_started":1}
W20260811 18:58:28.650979 13432 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:58:28.641251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.129:45423 (local address 127.29.112.190:44265)
0811 18:58:28.641471 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:28.641476 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:28.641833 (+ 357us) server_negotiation.cc:407] Connection header received
0811 18:58:28.642029 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:28.642033 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:28.642102 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:28.642183 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:28.643390 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:28.644160 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:28.645383 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:28.645798 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:28.645985 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:28.650700 (+ 4715us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.129:45423: BlockingRecv error: recv got EOF from 127.29.112.129:45423 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Aug 11 18:58:29 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.130@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:58:29 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.130@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:58:29.145323 13914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:29.130809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:44265 (local address 127.29.112.130:46045)
0811 18:58:29.131405 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:29.131428 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:29.131590 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:29.131954 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:29.131958 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:29.132724 (+ 766us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:58:29.132984 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:29.133000 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:29.137377 (+ 4377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:29.137382 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:29.138648 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:29.138739 (+ 91us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:29.139834 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:29.139843 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:29.140014 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:29.141011 (+ 997us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:58:29.141042 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:58:29.145086 (+ 4044us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.190:44265: Server oryx/127.29.112.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":292,"threads_started":1}
W20260811 18:58:29.145612 13432 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:58:29.130892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.130:46045 (local address 127.29.112.190:44265)
0811 18:58:29.131145 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:29.131153 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:29.131521 (+ 368us) server_negotiation.cc:407] Connection header received
0811 18:58:29.131719 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:29.131726 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:29.131807 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:29.131926 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:29.133284 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:29.137166 (+ 3882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:29.138959 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:29.139581 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:29.139725 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:29.145318 (+ 5593us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.130:46045: BlockingRecv error: recv got EOF from 127.29.112.130:46045 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Aug 11 18:58:29 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.131@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:58:29 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.131@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:58:29.627741 13915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:29.618381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:44265 (local address 127.29.112.131:44305)
0811 18:58:29.619231 (+ 850us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:29.619250 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:29.619429 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:29.620026 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:29.620031 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:29.620479 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:58:29.620731 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:29.620741 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:29.621755 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:29.621759 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:29.622707 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:29.622717 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:29.623672 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:29.623679 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:29.623884 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:29.624628 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:58:29.624644 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:58:29.627507 (+ 2863us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.190:44265: Server oryx/127.29.112.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":688,"thread_start_us":543,"threads_started":1}
W20260811 18:58:29.627952 13432 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:58:29.618496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.131:44305 (local address 127.29.112.190:44265)
0811 18:58:29.618916 (+ 420us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:29.618933 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:29.619351 (+ 418us) server_negotiation.cc:407] Connection header received
0811 18:58:29.619613 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:29.619618 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:29.619698 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:29.619812 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:29.620880 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:29.621616 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:29.622820 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:29.623403 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:29.623544 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:29.627667 (+ 4123us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.131:44305: BlockingRecv error: recv got EOF from 127.29.112.131:44305 (error 108)
Metrics: {"server-negotiator.queue_time_us":105}
Aug 11 18:58:29 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.129@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:58:29 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.129@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:58:29.660861 13916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:29.651778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:44265 (local address 127.29.112.129:59499)
0811 18:58:29.652391 (+ 613us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:29.652410 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:29.652543 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:29.652909 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:29.652912 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:29.653332 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:58:29.653507 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:29.653516 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:29.654633 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:29.654637 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:29.655816 (+ 1179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:29.655835 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:29.656838 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:29.656843 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:29.656971 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:29.657668 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:58:29.657686 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:58:29.660623 (+ 2937us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.190:44265: Server oryx/127.29.112.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":336,"threads_started":1}
W20260811 18:58:29.661128 13432 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:58:29.651865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.129:59499 (local address 127.29.112.190:44265)
0811 18:58:29.652039 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:29.652043 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:29.652519 (+ 476us) server_negotiation.cc:407] Connection header received
0811 18:58:29.652665 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:29.652670 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:29.652737 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:29.652826 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:29.653669 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:29.654449 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:29.655964 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:29.656676 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:29.656797 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:29.660815 (+ 4018us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.129:59499: BlockingRecv error: recv got EOF from 127.29.112.129:59499 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 11 18:58:30 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.130@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:58:30 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.130@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:58:30.159922 13917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:30.146329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:44265 (local address 127.29.112.130:35673)
0811 18:58:30.146935 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:30.146953 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:30.147064 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:30.147378 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:30.147381 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:30.147783 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:58:30.147976 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:30.147984 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:30.149043 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:30.149047 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:30.150031 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:30.150046 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:30.151505 (+ 1459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:30.151512 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:30.151684 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:30.156410 (+ 4726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:58:30.156429 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:58:30.159647 (+ 3218us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.190:44265: Server oryx/127.29.112.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":331,"threads_started":1}
W20260811 18:58:30.160211 13432 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:58:30.146398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.130:35673 (local address 127.29.112.190:44265)
0811 18:58:30.146676 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:30.146683 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:30.147029 (+ 346us) server_negotiation.cc:407] Connection header received
0811 18:58:30.147162 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:30.147166 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:30.147229 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:30.147313 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:30.148119 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:30.148900 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:30.150339 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:30.151338 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:30.151452 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:30.159927 (+ 8475us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.130:35673: BlockingRecv error: recv got EOF from 127.29.112.130:35673 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 11 18:58:30 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.131@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:58:30 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.131@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:58:30.637637 13918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:30.628838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:44265 (local address 127.29.112.131:47679)
0811 18:58:30.629339 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:30.629355 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:30.629473 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:30.629724 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:30.629727 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:30.630118 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:58:30.630281 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:30.630289 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:30.631370 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:30.631374 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:30.632296 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:30.632307 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:30.633070 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:30.633079 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:30.633285 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:30.633955 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:58:30.633973 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:58:30.637259 (+ 3286us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.190:44265: Server oryx/127.29.112.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":264,"threads_started":1}
W20260811 18:58:30.637678 13432 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:58:30.628843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.131:47679 (local address 127.29.112.190:44265)
0811 18:58:30.629080 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:30.629090 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:30.629436 (+ 346us) server_negotiation.cc:407] Connection header received
0811 18:58:30.629567 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:30.629570 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:30.629632 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:30.629711 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:30.630402 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:30.631250 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:30.632431 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:30.632864 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:30.633004 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:30.637448 (+ 4444us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.131:47679: BlockingRecv error: recv got EOF from 127.29.112.131:47679 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 11 18:58:30 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.129@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:58:30 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.129@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:58:30.670660 13919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:30.661754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:44265 (local address 127.29.112.129:48269)
0811 18:58:30.662316 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:30.662334 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:30.662454 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:30.662792 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:30.662796 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:30.663282 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:58:30.663444 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:30.663453 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:30.664576 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:30.664580 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:30.665597 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:30.665608 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:30.666289 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:30.666294 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:30.666415 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:30.667166 (+ 751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:58:30.667184 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:58:30.670331 (+ 3147us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.190:44265: Server oryx/127.29.112.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":341,"threads_started":1}
W20260811 18:58:30.670830 13432 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:58:30.661843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.129:48269 (local address 127.29.112.190:44265)
0811 18:58:30.662024 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:30.662029 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:30.662411 (+ 382us) server_negotiation.cc:407] Connection header received
0811 18:58:30.662597 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:30.662601 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:30.662662 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:30.662744 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:30.663604 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:30.664444 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:30.665729 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:30.666144 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:30.666242 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:30.670597 (+ 4355us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.129:48269: BlockingRecv error: recv got EOF from 127.29.112.129:48269 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 11 18:58:31 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.130@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:58:31 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.130@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:58:31.172051 13920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:31.160926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:44265 (local address 127.29.112.130:55513)
0811 18:58:31.161712 (+ 786us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:31.161741 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:31.161980 (+ 239us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:31.162686 (+ 706us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:31.162730 (+ 44us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:31.163294 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:58:31.163509 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:31.163519 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:31.164920 (+ 1401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:31.164927 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:31.165930 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:31.165943 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:31.166760 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:31.166766 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:31.166966 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:31.167705 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:58:31.167723 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:58:31.171400 (+ 3677us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.190:44265: Server oryx/127.29.112.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":560,"thread_start_us":392,"threads_started":1}
W20260811 18:58:31.172339 13432 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:58:31.161180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.130:55513 (local address 127.29.112.190:44265)
0811 18:58:31.161464 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:31.161472 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:31.161876 (+ 404us) server_negotiation.cc:407] Connection header received
0811 18:58:31.162214 (+ 338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:31.162295 (+ 81us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:31.162438 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:31.162654 (+ 216us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:31.163683 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:31.164707 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:31.166092 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:31.166606 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:31.166746 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:31.171893 (+ 5147us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.130:55513: BlockingRecv error: recv got EOF from 127.29.112.130:55513 (error 108)
Metrics: {"server-negotiator.queue_time_us":118}
Aug 11 18:58:31 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.131@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:58:31 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.131@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:58:31.649221 13921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:31.638707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:44265 (local address 127.29.112.131:56387)
0811 18:58:31.639246 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:31.639263 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:31.639374 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:31.639669 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:31.639672 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:31.640061 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:58:31.640206 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:31.640214 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:31.641760 (+ 1546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:31.641764 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:31.642797 (+ 1033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:31.642809 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:31.643765 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:31.643771 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:31.643943 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:31.644777 (+ 834us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:58:31.644806 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:58:31.648892 (+ 4086us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.190:44265: Server oryx/127.29.112.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":326,"threads_started":1}
W20260811 18:58:31.649435 13432 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:58:31.638898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.131:56387 (local address 127.29.112.190:44265)
0811 18:58:31.639111 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:31.639116 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:31.639340 (+ 224us) server_negotiation.cc:407] Connection header received
0811 18:58:31.639473 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:31.639476 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:31.639543 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:31.639632 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:31.640556 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:31.641539 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:31.642980 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:31.643560 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:31.643690 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:31.649150 (+ 5460us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.131:56387: BlockingRecv error: recv got EOF from 127.29.112.131:56387 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 11 18:58:31 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.129@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:58:31 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.129@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:58:31.681777 13922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:31.671876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:44265 (local address 127.29.112.129:58579)
0811 18:58:31.672411 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:31.672432 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:31.672573 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:31.673149 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:31.673152 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:31.673562 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:58:31.673740 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:31.673749 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:31.674980 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:31.674986 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:31.676436 (+ 1450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:31.676459 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:31.677379 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:31.677384 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:31.677556 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:31.678421 (+ 865us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:58:31.678440 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:58:31.681523 (+ 3083us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.190:44265: Server oryx/127.29.112.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":431,"threads_started":1}
W20260811 18:58:31.682087 13432 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:58:31.671932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.129:58579 (local address 127.29.112.190:44265)
0811 18:58:31.672146 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:31.672153 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:31.672819 (+ 666us) server_negotiation.cc:407] Connection header received
0811 18:58:31.672889 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:31.672896 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:31.672984 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:31.673106 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:31.673889 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:31.674747 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:31.676740 (+ 1993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:31.677173 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:31.677340 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:31.681756 (+ 4416us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.129:58579: BlockingRecv error: recv got EOF from 127.29.112.129:58579 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Aug 11 18:58:32 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.130@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:58:32 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.130@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:58:32.182426 13923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:32.172924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:44265 (local address 127.29.112.130:51853)
0811 18:58:32.173679 (+ 755us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:32.173706 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:32.173900 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:32.174308 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:32.174312 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:32.174849 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:58:32.175052 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:32.175061 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:32.176431 (+ 1370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:32.176444 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:32.177659 (+ 1215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:32.177671 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:32.178624 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:32.178630 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:32.178756 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:32.179355 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:58:32.179373 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:58:32.182190 (+ 2817us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.190:44265: Server oryx/127.29.112.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":571,"thread_start_us":258,"threads_started":1}
W20260811 18:58:32.182771 13432 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:58:32.173129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.130:51853 (local address 127.29.112.190:44265)
0811 18:58:32.173367 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:32.173376 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:32.173847 (+ 471us) server_negotiation.cc:407] Connection header received
0811 18:58:32.174045 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:32.174049 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:32.174119 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:32.174212 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:32.175210 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:32.176232 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:32.177809 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:32.178386 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:32.178484 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:32.182437 (+ 3953us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.130:51853: BlockingRecv error: recv got EOF from 127.29.112.130:51853 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
Aug 11 18:58:32 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.131@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:58:32 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.131@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:58:32.659364 13927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:32.650357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:44265 (local address 127.29.112.131:57017)
0811 18:58:32.650887 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:32.650903 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:32.651076 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:32.651410 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:32.651414 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:32.651848 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:58:32.652027 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:32.652036 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:32.653175 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:32.653181 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:32.654285 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:32.654298 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:32.655109 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:32.655115 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:32.655297 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:32.656030 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:58:32.656067 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:58:32.659121 (+ 3054us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.190:44265: Server oryx/127.29.112.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":294,"threads_started":1}
W20260811 18:58:32.659574 13432 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:58:32.650439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.131:57017 (local address 127.29.112.190:44265)
0811 18:58:32.650738 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:32.650746 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:32.651001 (+ 255us) server_negotiation.cc:407] Connection header received
0811 18:58:32.651198 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:32.651203 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:32.651274 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:32.651380 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:32.652174 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:32.653036 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:32.654457 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:32.654951 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:32.655089 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:32.659314 (+ 4225us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.131:57017: BlockingRecv error: recv got EOF from 127.29.112.131:57017 (error 108)
Metrics: {"server-negotiator.queue_time_us":117}
Aug 11 18:58:32 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.129@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:58:32 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.129@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:58:32.692020 13929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:32.682809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:44265 (local address 127.29.112.129:60705)
0811 18:58:32.683411 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:32.683427 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:32.683541 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:32.683894 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:32.683898 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:32.684304 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:58:32.684467 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:32.684475 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:32.685677 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:32.685681 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:32.686780 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:32.686792 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:32.687501 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:32.687505 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:32.687646 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:32.688343 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:58:32.688360 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:58:32.691681 (+ 3321us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.190:44265: Server oryx/127.29.112.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":341,"threads_started":1}
W20260811 18:58:32.692137 13432 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:58:32.682888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.129:60705 (local address 127.29.112.190:44265)
0811 18:58:32.683095 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:32.683100 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:32.683578 (+ 478us) server_negotiation.cc:407] Connection header received
0811 18:58:32.683640 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:32.683644 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:32.683711 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:32.683805 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:32.684656 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:32.685493 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:32.686926 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:32.687364 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:32.687464 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:32.691884 (+ 4420us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.129:60705: BlockingRecv error: recv got EOF from 127.29.112.129:60705 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 11 18:58:33 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.130@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:58:33 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.130@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:58:33.193275 13931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:33.183775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:44265 (local address 127.29.112.130:37493)
0811 18:58:33.184498 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:33.184520 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:33.184680 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:33.185111 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:33.185117 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:33.185707 (+ 590us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:58:33.185899 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:33.185908 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:33.187292 (+ 1384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:33.187296 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:33.188219 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:33.188230 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:33.188978 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:33.188983 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:33.189123 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:33.189780 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:58:33.189799 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:58:33.192927 (+ 3128us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.190:44265: Server oryx/127.29.112.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":602,"thread_start_us":445,"threads_started":1}
W20260811 18:58:33.193392 13432 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:58:33.183873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.130:37493 (local address 127.29.112.190:44265)
0811 18:58:33.184132 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:33.184144 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:33.184626 (+ 482us) server_negotiation.cc:407] Connection header received
0811 18:58:33.184845 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:33.184852 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:33.184946 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:33.185043 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:33.186150 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:33.187135 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:33.188373 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:33.188801 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:33.188902 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:33.193152 (+ 4250us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.130:37493: BlockingRecv error: recv got EOF from 127.29.112.130:37493 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 11 18:58:33 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.131@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:58:33 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.131@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:58:33.669142 13932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:33.660405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:44265 (local address 127.29.112.131:59245)
0811 18:58:33.661243 (+ 838us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:33.661267 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:33.661484 (+ 217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:33.661793 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:33.661796 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:33.662249 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:58:33.662494 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:33.662513 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:33.663693 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:33.663698 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:33.664620 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:33.664633 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:33.665351 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:33.665355 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:33.665497 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:33.666082 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:58:33.666100 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:58:33.668910 (+ 2810us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.190:44265: Server oryx/127.29.112.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":689,"thread_start_us":537,"threads_started":1}
W20260811 18:58:33.669339 13432 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:58:33.660439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.131:59245 (local address 127.29.112.190:44265)
0811 18:58:33.660708 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:33.660714 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:33.661424 (+ 710us) server_negotiation.cc:407] Connection header received
0811 18:58:33.661596 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:33.661600 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:33.661663 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:33.661753 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:33.662699 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:33.663497 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:33.664748 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:33.665213 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:33.665303 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:33.669098 (+ 3795us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.131:59245: BlockingRecv error: recv got EOF from 127.29.112.131:59245 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 11 18:58:33 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.129@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:58:33 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.129@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:58:33.702162 13934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:33.693180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:44265 (local address 127.29.112.129:49337)
0811 18:58:33.693695 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:33.693711 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:33.693829 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:33.694137 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:33.694140 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:33.694662 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:58:33.694871 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:33.694881 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:33.695938 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:33.695946 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:33.696885 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:33.696896 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:33.697651 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:33.697666 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:33.697832 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:33.698437 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:58:33.698454 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:58:33.701909 (+ 3455us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.190:44265: Server oryx/127.29.112.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":272,"threads_started":1}
W20260811 18:58:33.702373 13432 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:58:33.693239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.129:49337 (local address 127.29.112.190:44265)
0811 18:58:33.693462 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:33.693472 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:33.693810 (+ 338us) server_negotiation.cc:407] Connection header received
0811 18:58:33.693934 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:33.693938 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:33.693999 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:33.694088 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:33.695024 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:33.695784 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:33.697047 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:33.697456 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:33.697594 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:33.702106 (+ 4512us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.129:49337: BlockingRecv error: recv got EOF from 127.29.112.129:49337 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 11 18:58:34 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.130@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:58:34 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.130@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:58:34.202685 13935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:34.194436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:44265 (local address 127.29.112.130:39599)
0811 18:58:34.195016 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:34.195033 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:34.195147 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:34.195468 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:34.195472 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:34.195888 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:58:34.196044 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:34.196052 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:34.197120 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:34.197124 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:34.198038 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:34.198049 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:34.198780 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:34.198785 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:34.198917 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:34.199579 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:58:34.199596 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:58:34.202369 (+ 2773us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.190:44265: Server oryx/127.29.112.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":317,"threads_started":1}
W20260811 18:58:34.202973 13432 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:58:34.194702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.130:39599 (local address 127.29.112.190:44265)
0811 18:58:34.194922 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:34.194931 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:34.195110 (+ 179us) server_negotiation.cc:407] Connection header received
0811 18:58:34.195248 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:34.195252 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:34.195330 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:34.195418 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:34.196197 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:34.196969 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:34.198180 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:34.198645 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:34.198741 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:34.202687 (+ 3946us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.130:39599: BlockingRecv error: recv got EOF from 127.29.112.130:39599 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 11 18:58:34 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.131@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:58:34 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.131@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:58:34.680650 13938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:34.670172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:44265 (local address 127.29.112.131:49663)
0811 18:58:34.670805 (+ 633us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:34.670821 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:34.671138 (+ 317us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:34.671747 (+ 609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:34.671751 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:34.672229 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:58:34.672474 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:34.672485 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:34.673969 (+ 1484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:34.673976 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:34.675018 (+ 1042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:34.675031 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:34.675993 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:34.675998 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:34.676119 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:34.676868 (+ 749us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:58:34.676886 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:58:34.680407 (+ 3521us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.190:44265: Server oryx/127.29.112.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":527,"thread_start_us":372,"threads_started":1}
W20260811 18:58:34.680924 13432 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:58:34.670256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.131:49663 (local address 127.29.112.190:44265)
0811 18:58:34.670493 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:34.670498 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:34.670941 (+ 443us) server_negotiation.cc:407] Connection header received
0811 18:58:34.671445 (+ 504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:34.671451 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:34.671539 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:34.671650 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:34.672665 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:34.673762 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:34.675241 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:34.675783 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:34.675920 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:34.680665 (+ 4745us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.131:49663: BlockingRecv error: recv got EOF from 127.29.112.131:49663 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
Aug 11 18:58:34 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.129@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:58:34 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.129@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:58:34.712494 13941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:34.703286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:44265 (local address 127.29.112.129:36443)
0811 18:58:34.703795 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:34.703812 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:34.703931 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:34.704492 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:34.704495 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:34.704890 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:58:34.705184 (+ 294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:34.705194 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:34.706333 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:34.706337 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:34.707339 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:34.707351 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:34.708175 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:34.708179 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:34.708303 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:34.709086 (+ 783us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:58:34.709114 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:58:34.712240 (+ 3126us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.190:44265: Server oryx/127.29.112.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":287,"threads_started":1}
W20260811 18:58:34.712761 13432 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:58:34.703368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.129:36443 (local address 127.29.112.190:44265)
0811 18:58:34.703635 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:34.703640 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:34.703887 (+ 247us) server_negotiation.cc:407] Connection header received
0811 18:58:34.704163 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:34.704169 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:34.704289 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:34.704416 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:34.705374 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:34.706189 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:34.707469 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:34.707990 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:34.708123 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:34.712469 (+ 4346us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.129:36443: BlockingRecv error: recv got EOF from 127.29.112.129:36443 (error 108)
Metrics: {"server-negotiator.queue_time_us":125}
Aug 11 18:58:35 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.130@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:58:35 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.130@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:58:35.217505 13942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:35.203682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:44265 (local address 127.29.112.130:47301)
0811 18:58:35.204447 (+ 765us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:35.204464 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:35.204661 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:35.205220 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:35.205224 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:35.205750 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:58:35.206034 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:35.206045 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:35.207215 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:35.207221 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:35.208191 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:35.208202 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:35.208919 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:35.208923 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:35.209050 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:35.214295 (+ 5245us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:58:35.214313 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:58:35.217133 (+ 2820us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.190:44265: Server oryx/127.29.112.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":627,"thread_start_us":471,"threads_started":1}
W20260811 18:58:35.217679 13432 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:58:35.203874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.130:47301 (local address 127.29.112.190:44265)
0811 18:58:35.204306 (+ 432us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:35.204313 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:35.204581 (+ 268us) server_negotiation.cc:407] Connection header received
0811 18:58:35.204798 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:35.204803 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:35.205046 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:35.205193 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:35.206179 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:35.207061 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:35.208331 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:35.208789 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:35.208885 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:35.217379 (+ 8494us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.130:47301: BlockingRecv error: recv got EOF from 127.29.112.130:47301 (error 108)
Metrics: {"server-negotiator.queue_time_us":279}
Aug 11 18:58:35 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.131@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:58:35 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.131@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:58:35.690919 13944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:35.681701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:44265 (local address 127.29.112.131:51279)
0811 18:58:35.682231 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:35.682246 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:35.682415 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:35.682771 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:35.682776 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:35.683207 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:58:35.683384 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:35.683393 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:35.684461 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:35.684471 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:35.685488 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:35.685506 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:35.686620 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:35.686627 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:35.686837 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:35.687580 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:58:35.687597 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:58:35.690648 (+ 3051us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.190:44265: Server oryx/127.29.112.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":304,"threads_started":1}
W20260811 18:58:35.691082 13432 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:58:35.681794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.131:51279 (local address 127.29.112.190:44265)
0811 18:58:35.682037 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:35.682046 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:35.682334 (+ 288us) server_negotiation.cc:407] Connection header received
0811 18:58:35.682562 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:35.682566 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:35.682629 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:35.682738 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:35.683531 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:35.684297 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:35.685767 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:35.686438 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:35.686603 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:35.690849 (+ 4246us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.131:51279: BlockingRecv error: recv got EOF from 127.29.112.131:51279 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 11 18:58:35 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.129@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:58:35 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.129@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:58:35.722711 13945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:35.713673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:44265 (local address 127.29.112.129:56501)
0811 18:58:35.714362 (+ 689us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:35.714381 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:35.714522 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:35.714874 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:35.714877 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:35.715305 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:58:35.715475 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:35.715483 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:35.716560 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:35.716564 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:35.717507 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:35.717518 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:35.718416 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:35.718420 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:35.718572 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:35.719299 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:58:35.719315 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:58:35.722267 (+ 2952us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.190:44265: Server oryx/127.29.112.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":370,"threads_started":1}
W20260811 18:58:35.722756 13432 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:58:35.713776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.129:56501 (local address 127.29.112.190:44265)
0811 18:58:35.713978 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:35.713983 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:35.714488 (+ 505us) server_negotiation.cc:407] Connection header received
0811 18:58:35.714657 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:35.714662 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:35.714730 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:35.714830 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:35.715633 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:35.716411 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:35.717643 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:35.718255 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:35.718358 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:35.722548 (+ 4190us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.129:56501: BlockingRecv error: recv got EOF from 127.29.112.129:56501 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 11 18:58:36 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.130@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:58:36 dist-test-slave-h6n0 krb5kdc[12231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.112.130@KRBTEST.COM for oryx/127.29.112.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:58:36.226963 13946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:36.218809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:44265 (local address 127.29.112.130:49727)
0811 18:58:36.219294 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:36.219311 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:36.219420 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:36.219761 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:36.219765 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:36.220195 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:58:36.220364 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:36.220372 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:36.221349 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:36.221352 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:36.222281 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:36.222292 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:36.222978 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:36.222982 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:36.223168 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:36.223810 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:58:36.223840 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:58:36.226710 (+ 2870us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.112.190:44265: Server oryx/127.29.112.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":282,"threads_started":1}
W20260811 18:58:36.227104 13432 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:58:36.218856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.130:49727 (local address 127.29.112.190:44265)
0811 18:58:36.219082 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:36.219092 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:36.219391 (+ 299us) server_negotiation.cc:407] Connection header received
0811 18:58:36.219526 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:36.219529 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:36.219591 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:36.219671 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:36.220498 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:36.221213 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:36.222407 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:36.222841 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:36.222933 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:36.226893 (+ 3960us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.112.130:49727: BlockingRecv error: recv got EOF from 127.29.112.130:49727 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 18:58:36.523710 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 13183
I20260811 18:58:36.536769 13950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:36.525300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56248 (local address 127.29.112.129:45315)
0811 18:58:36.526019 (+ 719us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:36.526027 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:36.526052 (+ 25us) server_negotiation.cc:407] Connection header received
0811 18:58:36.526116 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:36.526122 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:36.526331 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:36.526571 (+ 240us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:36.528047 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:36.529005 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:36.530198 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:36.530829 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:36.530970 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:36.533576 (+ 2606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:58:36.533585 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:58:36.533598 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:58:36.533628 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:58:36.536225 (+ 2597us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.29.112.129@KRBTEST.COM not found in keytab (ticket kvno 2)
0811 18:58:36.536553 (+ 328us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:56248: Request ticket server kudu/127.29.112.129@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":552,"thread_start_us":402,"threads_started":1}
W20260811 18:58:36.536903 13950 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:56248: Request ticket server kudu/127.29.112.129@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260811 18:58:36.536914 30146 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 13183: Not authorized: Client connection negotiation failed: client connection to 127.29.112.129:45315: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.29.112.129@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260811 18:58:36.537101 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 13183
I20260811 18:58:36.563462 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 13431
I20260811 18:58:36.577610 13952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:36.564654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58784 (local address 127.29.112.130:43559)
0811 18:58:36.565302 (+ 648us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:36.565310 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:36.565341 (+ 31us) server_negotiation.cc:407] Connection header received
0811 18:58:36.565413 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:36.565420 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:36.565630 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:36.565847 (+ 217us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:36.567328 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:36.568416 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:36.569681 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:36.570217 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:36.570367 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:36.573605 (+ 3238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:58:36.573617 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:58:36.573636 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:58:36.573698 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:58:36.576817 (+ 3119us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.29.112.130@KRBTEST.COM not found in keytab (ticket kvno 2)
0811 18:58:36.577257 (+ 440us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:58784: Request ticket server kudu/127.29.112.130@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":501,"thread_start_us":351,"threads_started":1}
W20260811 18:58:36.577700 30146 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 13431: Not authorized: Client connection negotiation failed: client connection to 127.29.112.130:43559: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.29.112.130@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260811 18:58:36.577764 13952 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:58784: Request ticket server kudu/127.29.112.130@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260811 18:58:36.577875 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 13431
I20260811 18:58:36.603399 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 13646
I20260811 18:58:36.616024 13953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:36.604558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57264 (local address 127.29.112.131:39027)
0811 18:58:36.605108 (+ 550us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:36.605116 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:36.605141 (+ 25us) server_negotiation.cc:407] Connection header received
0811 18:58:36.605201 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:36.605207 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:36.605412 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:36.605626 (+ 214us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:36.607070 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:36.608157 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:36.609381 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:36.609931 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:36.610074 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:36.612722 (+ 2648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:58:36.612731 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:58:36.612743 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:58:36.612779 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:58:36.615338 (+ 2559us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.29.112.131@KRBTEST.COM not found in keytab (ticket kvno 2)
0811 18:58:36.615667 (+ 329us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:57264: Request ticket server kudu/127.29.112.131@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":417,"thread_start_us":273,"threads_started":1}
W20260811 18:58:36.616158 13953 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:57264: Request ticket server kudu/127.29.112.131@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260811 18:58:36.616194 30146 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 13646: Not authorized: Client connection negotiation failed: client connection to 127.29.112.131:39027: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.29.112.131@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260811 18:58:36.616335 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 13646
I20260811 18:58:36.646070 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 13074
I20260811 18:58:36.647636 13175 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:58:36.767560 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 13074
2026-08-11T18:58:36Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (21970 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-11T18:58:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:58:36Z Disabled control of system clock
I20260811 18:58:36.868755 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474615205622-30146-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474615205622-30146-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474615205622-30146-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.190:38859
--webserver_interface=127.29.112.190
--webserver_port=0
--builtin_ntp_servers=127.29.112.148:43061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.190:38859
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260811 18:58:37.219977 13960 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:58:37.220265 13960 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:58:37.220403 13960 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:58:37.228150 13960 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:58:37.228266 13960 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:58:37.228305 13960 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:58:37.228421 13960 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:58:37.228484 13960 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:58:37.237751 13960 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:43061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474615205622-30146-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474615205622-30146-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.112.190:38859
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.190:38859
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.112.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474615205622-30146-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:58:37.239054 13960 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:58:37.240804 13960 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:58:37.252261 13966 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:37.252606 13965 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:37.252519 13970 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:58:37.253681 13960 server_base.cc:1034] running on GCE node
I20260811 18:58:37.254487 13960 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:58:37.255852 13960 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:58:37.257083 13960 hybrid_clock.cc:648] HybridClock initialized: now 1786474717257034 us; error 46 us; skew 500 ppm
I20260811 18:58:37.257416 13960 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 18:58:37.259820 13960 webserver.cc:460] Webserver started at http://127.29.112.190:41265/ using document root <none> and password file <none>
I20260811 18:58:37.260515 13960 fs_manager.cc:359] Metadata directory not provided
I20260811 18:58:37.260650 13960 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:58:37.260982 13960 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:58:37.263996 13960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474615205622-30146-0/minicluster-data/master-0/data/instance:
uuid: "70019181b59541a783e4721c15eca977"
format_stamp: "Formatted at 2026-08-11 18:58:37 on dist-test-slave-h6n0"
server_key: "48ed9f3168853d8966e51df027fd95e2"
server_key_iv: "78c8fdfcd8fc74c7ae06b7ba3153248d"
server_key_version: "encryptionkey@0"
I20260811 18:58:37.264837 13960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474615205622-30146-0/minicluster-data/master-0/wal/instance:
uuid: "70019181b59541a783e4721c15eca977"
format_stamp: "Formatted at 2026-08-11 18:58:37 on dist-test-slave-h6n0"
server_key: "48ed9f3168853d8966e51df027fd95e2"
server_key_iv: "78c8fdfcd8fc74c7ae06b7ba3153248d"
server_key_version: "encryptionkey@0"
I20260811 18:58:37.269872 13960 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260811 18:58:37.273494 13983 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:37.275049 13960 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 18:58:37.275224 13960 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474615205622-30146-0/minicluster-data/master-0/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474615205622-30146-0/minicluster-data/master-0/wal
uuid: "70019181b59541a783e4721c15eca977"
format_stamp: "Formatted at 2026-08-11 18:58:37 on dist-test-slave-h6n0"
server_key: "48ed9f3168853d8966e51df027fd95e2"
server_key_iv: "78c8fdfcd8fc74c7ae06b7ba3153248d"
server_key_version: "encryptionkey@0"
I20260811 18:58:37.275444 13960 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474615205622-30146-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474615205622-30146-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474615205622-30146-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:58:37.294873 13960 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:58:37.295763 13960 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:58:37.296087 13960 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:58:37.317452 13960 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.190:38859
I20260811 18:58:37.317471 14062 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.190:38859 every 8 connection(s)
I20260811 18:58:37.318908 13960 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
I20260811 18:58:37.323048 14063 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:58:37.324324 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 13960
I20260811 18:58:37.324575 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474615205622-30146-0/minicluster-data/master-0/wal/instance
I20260811 18:58:37.325459 30146 external_mini_cluster.cc:1428] Setting key 62c7b51b42af17a34ccf37da0dd7bfc8
I20260811 18:58:37.337666 14063 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70019181b59541a783e4721c15eca977: Bootstrap starting.
I20260811 18:58:37.339555 14067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:37.327636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54638 (local address 127.29.112.190:38859)
0811 18:58:37.328748 (+ 1112us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:37.328759 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:37.328802 (+ 43us) server_negotiation.cc:407] Connection header received
0811 18:58:37.330385 (+ 1583us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:37.330405 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:37.330946 (+ 541us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:37.331272 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:37.331899 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:37.333128 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:37.335123 (+ 1995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:37.335778 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:37.335923 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:37.337189 (+ 1266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:58:37.337196 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:58:37.337208 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:58:37.337758 (+ 550us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:58:37.337864 (+ 106us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:58:37.337887 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:58:37.337990 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:37.338219 (+ 229us) server_negotiation.cc:299] Negotiation successful
0811 18:58:37.338629 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":641,"thread_start_us":413,"threads_started":1}
I20260811 18:58:37.341007 14063 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 70019181b59541a783e4721c15eca977: Neither blocks nor log segments found. Creating new log.
I20260811 18:58:37.342142 14063 log.cc:824] T 00000000000000000000000000000000 P 70019181b59541a783e4721c15eca977: Log is configured to *not* fsync() on all Append() calls
I20260811 18:58:37.345039 14063 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70019181b59541a783e4721c15eca977: No bootstrap required, opened a new log
I20260811 18:58:37.352691 14063 raft_consensus.cc:348] T 00000000000000000000000000000000 P 70019181b59541a783e4721c15eca977 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70019181b59541a783e4721c15eca977" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 38859 } }
I20260811 18:58:37.353070 14063 raft_consensus.cc:374] T 00000000000000000000000000000000 P 70019181b59541a783e4721c15eca977 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:58:37.353188 14063 raft_consensus.cc:729] T 00000000000000000000000000000000 P 70019181b59541a783e4721c15eca977 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70019181b59541a783e4721c15eca977, State: Initialized, Role: FOLLOWER
I20260811 18:58:37.353832 14063 consensus_queue.cc:260] T 00000000000000000000000000000000 P 70019181b59541a783e4721c15eca977 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70019181b59541a783e4721c15eca977" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 38859 } }
I20260811 18:58:37.354071 14063 raft_consensus.cc:388] T 00000000000000000000000000000000 P 70019181b59541a783e4721c15eca977 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:58:37.354223 14063 raft_consensus.cc:482] T 00000000000000000000000000000000 P 70019181b59541a783e4721c15eca977 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:58:37.354404 14063 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 70019181b59541a783e4721c15eca977 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:58:37.356549 14063 raft_consensus.cc:504] T 00000000000000000000000000000000 P 70019181b59541a783e4721c15eca977 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70019181b59541a783e4721c15eca977" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 38859 } }
I20260811 18:58:37.357059 14063 leader_election.cc:302] T 00000000000000000000000000000000 P 70019181b59541a783e4721c15eca977 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 70019181b59541a783e4721c15eca977; no voters:
I20260811 18:58:37.357635 14063 leader_election.cc:288] T 00000000000000000000000000000000 P 70019181b59541a783e4721c15eca977 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:58:37.357806 14070 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 70019181b59541a783e4721c15eca977 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:58:37.358570 14070 raft_consensus.cc:686] T 00000000000000000000000000000000 P 70019181b59541a783e4721c15eca977 [term 1 LEADER]: Becoming Leader. State: Replica: 70019181b59541a783e4721c15eca977, State: Running, Role: LEADER
I20260811 18:58:37.359163 14070 consensus_queue.cc:237] T 00000000000000000000000000000000 P 70019181b59541a783e4721c15eca977 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70019181b59541a783e4721c15eca977" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 38859 } }
I20260811 18:58:37.359820 14063 sys_catalog.cc:564] T 00000000000000000000000000000000 P 70019181b59541a783e4721c15eca977 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:58:37.362298 14074 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70019181b59541a783e4721c15eca977 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 70019181b59541a783e4721c15eca977. Latest consensus state: current_term: 1 leader_uuid: "70019181b59541a783e4721c15eca977" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70019181b59541a783e4721c15eca977" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 38859 } } }
I20260811 18:58:37.362803 14074 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70019181b59541a783e4721c15eca977 [sys.catalog]: This master's current role is: LEADER
I20260811 18:58:37.362769 14071 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70019181b59541a783e4721c15eca977 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "70019181b59541a783e4721c15eca977" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70019181b59541a783e4721c15eca977" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 38859 } } }
I20260811 18:58:37.363121 14071 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70019181b59541a783e4721c15eca977 [sys.catalog]: This master's current role is: LEADER
I20260811 18:58:37.364853 14085 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:58:37.369500 14085 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:58:37.377532 14085 catalog_manager.cc:1347] Generated new cluster ID: 23c00c28732b462695b9e9e85161ba0f
I20260811 18:58:37.377653 14085 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:58:37.387089 14085 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:58:37.388298 14085 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:58:37.401368 14085 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 70019181b59541a783e4721c15eca977: Generated new TSK 0
I20260811 18:58:37.402141 14085 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:58:37.411051 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.129:0
--local_ip_for_outbound_sockets=127.29.112.129
--webserver_interface=127.29.112.129
--webserver_port=0
--tserver_master_addrs=127.29.112.190:38859
--builtin_ntp_servers=127.29.112.148:43061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260811 18:58:37.755214 14099 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:58:37.755483 14099 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:58:37.755635 14099 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:58:37.762909 14099 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:58:37.763051 14099 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:58:37.763211 14099 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.129
I20260811 18:58:37.772541 14099 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:43061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.112.129
--webserver_port=0
--tserver_master_addrs=127.29.112.190:38859
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.129
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:58:37.773874 14099 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:58:37.775684 14099 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:58:37.789925 14113 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:37.789935 14118 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:37.790002 14114 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:58:37.791272 14099 server_base.cc:1034] running on GCE node
I20260811 18:58:37.791970 14099 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:58:37.793032 14099 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:58:37.794288 14099 hybrid_clock.cc:648] HybridClock initialized: now 1786474717794245 us; error 64 us; skew 500 ppm
I20260811 18:58:37.794696 14099 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 18:58:37.797158 14099 webserver.cc:460] Webserver started at http://127.29.112.129:44205/ using document root <none> and password file <none>
I20260811 18:58:37.797900 14099 fs_manager.cc:359] Metadata directory not provided
I20260811 18:58:37.798051 14099 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:58:37.798409 14099 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:58:37.801529 14099 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-0/data/instance:
uuid: "6274acc71d2244e489d2191a1a1910a3"
format_stamp: "Formatted at 2026-08-11 18:58:37 on dist-test-slave-h6n0"
server_key: "542067c1f0656cc659f1047cd1fe3801"
server_key_iv: "b5f4f149a68cef9471e768aca251b4aa"
server_key_version: "encryptionkey@0"
I20260811 18:58:37.802352 14099 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-0/wal/instance:
uuid: "6274acc71d2244e489d2191a1a1910a3"
format_stamp: "Formatted at 2026-08-11 18:58:37 on dist-test-slave-h6n0"
server_key: "542067c1f0656cc659f1047cd1fe3801"
server_key_iv: "b5f4f149a68cef9471e768aca251b4aa"
server_key_version: "encryptionkey@0"
I20260811 18:58:37.807680 14099 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 18:58:37.811203 14125 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:37.813033 14099 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 18:58:37.813321 14099 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-0/wal
uuid: "6274acc71d2244e489d2191a1a1910a3"
format_stamp: "Formatted at 2026-08-11 18:58:37 on dist-test-slave-h6n0"
server_key: "542067c1f0656cc659f1047cd1fe3801"
server_key_iv: "b5f4f149a68cef9471e768aca251b4aa"
server_key_version: "encryptionkey@0"
I20260811 18:58:37.813556 14099 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:58:37.833493 14099 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:58:37.834311 14099 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:58:37.834658 14099 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:58:37.835597 14099 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:58:37.837069 14099 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:58:37.837208 14099 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:37.837332 14099 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:58:37.837425 14099 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:37.873097 14099 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.129:33023
I20260811 18:58:37.873189 14320 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.129:33023 every 8 connection(s)
I20260811 18:58:37.874397 14099 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-0/data/info.pb
I20260811 18:58:37.882331 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 14099
I20260811 18:58:37.882568 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-0/wal/instance
I20260811 18:58:37.883292 30146 external_mini_cluster.cc:1428] Setting key 7e0a4debda4f46ec73db2e56fbd4122b
I20260811 18:58:37.886380 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.130:0
--local_ip_for_outbound_sockets=127.29.112.130
--webserver_interface=127.29.112.130
--webserver_port=0
--tserver_master_addrs=127.29.112.190:38859
--builtin_ntp_servers=127.29.112.148:43061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260811 18:58:37.888289 14323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:37.876849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.129:41597 (local address 127.29.112.190:38859)
0811 18:58:37.877306 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:37.877312 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:37.878449 (+ 1137us) server_negotiation.cc:407] Connection header received
0811 18:58:37.880468 (+ 2019us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:37.880473 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:37.880541 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:37.880644 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:37.881589 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:37.882751 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:37.884776 (+ 2025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:37.885231 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:37.885349 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:37.887187 (+ 1838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:58:37.887197 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:58:37.887205 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:58:37.887229 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:58:37.887282 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:58:37.887301 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:58:37.887398 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:37.887612 (+ 214us) server_negotiation.cc:299] Negotiation successful
0811 18:58:37.887722 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":217,"threads_started":1}
I20260811 18:58:37.888911 14325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:37.877457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:38859 (local address 127.29.112.129:41597)
0811 18:58:37.878301 (+ 844us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:37.878345 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:37.880212 (+ 1867us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:37.880820 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:37.880826 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:37.880899 (+ 73us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:58:37.881433 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:37.881448 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:37.882905 (+ 1457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:37.882909 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:37.884505 (+ 1596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:37.884535 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:37.885493 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:37.885501 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:37.885789 (+ 288us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:37.886852 (+ 1063us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:58:37.886855 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:58:37.886894 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:58:37.886974 (+ 80us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:58:37.887414 (+ 440us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:58:37.887422 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:58:37.887428 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 18:58:37.887561 (+ 133us) client_negotiation.cc:240] Negotiation successful
0811 18:58:37.887850 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":268,"threads_started":1}
I20260811 18:58:37.890440 14321 heartbeater.cc:347] Connected to a master server at 127.29.112.190:38859
I20260811 18:58:37.890785 14321 heartbeater.cc:464] Registering TS with master...
I20260811 18:58:37.891662 14321 heartbeater.cc:511] Master 127.29.112.190:38859 requested a full tablet report, sending...
I20260811 18:58:37.893760 14010 ts_manager.cc:194] Registered new tserver with Master: 6274acc71d2244e489d2191a1a1910a3 (127.29.112.129:33023)
I20260811 18:58:37.895345 14010 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.112.129:41597
W20260811 18:58:38.232769 14326 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:58:38.233065 14326 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:58:38.233155 14326 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:58:38.240342 14326 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:58:38.240451 14326 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:58:38.240558 14326 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.130
I20260811 18:58:38.250258 14326 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:43061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.112.130
--webserver_port=0
--tserver_master_addrs=127.29.112.190:38859
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.130
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:58:38.251672 14326 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:58:38.253527 14326 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:58:38.267860 14337 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:38.268036 14334 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:38.267890 14335 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:58:38.269745 14326 server_base.cc:1034] running on GCE node
I20260811 18:58:38.270393 14326 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:58:38.276243 14326 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:58:38.277519 14326 hybrid_clock.cc:648] HybridClock initialized: now 1786474718277435 us; error 64 us; skew 500 ppm
I20260811 18:58:38.277951 14326 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 18:58:38.281533 14326 webserver.cc:460] Webserver started at http://127.29.112.130:35449/ using document root <none> and password file <none>
I20260811 18:58:38.282617 14326 fs_manager.cc:359] Metadata directory not provided
I20260811 18:58:38.282719 14326 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:58:38.283017 14326 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:58:38.285933 14326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-1/data/instance:
uuid: "00dcb614480749cc8cfa6f79f88aa74d"
format_stamp: "Formatted at 2026-08-11 18:58:38 on dist-test-slave-h6n0"
server_key: "b0ebc6290b9dbf45540c2ddfdc54bf54"
server_key_iv: "5a152d47e2987e7453c01c89a472c67f"
server_key_version: "encryptionkey@0"
I20260811 18:58:38.286878 14326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-1/wal/instance:
uuid: "00dcb614480749cc8cfa6f79f88aa74d"
format_stamp: "Formatted at 2026-08-11 18:58:38 on dist-test-slave-h6n0"
server_key: "b0ebc6290b9dbf45540c2ddfdc54bf54"
server_key_iv: "5a152d47e2987e7453c01c89a472c67f"
server_key_version: "encryptionkey@0"
I20260811 18:58:38.292659 14326 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 18:58:38.296305 14346 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:38.298039 14326 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 18:58:38.298269 14326 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-1/wal
uuid: "00dcb614480749cc8cfa6f79f88aa74d"
format_stamp: "Formatted at 2026-08-11 18:58:38 on dist-test-slave-h6n0"
server_key: "b0ebc6290b9dbf45540c2ddfdc54bf54"
server_key_iv: "5a152d47e2987e7453c01c89a472c67f"
server_key_version: "encryptionkey@0"
I20260811 18:58:38.298516 14326 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:58:38.330142 14326 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:58:38.331115 14326 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:58:38.331427 14326 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:58:38.332440 14326 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:58:38.333945 14326 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:58:38.334087 14326 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:38.334223 14326 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:58:38.334316 14326 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:38.372290 14326 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.130:35485
I20260811 18:58:38.372411 14523 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.130:35485 every 8 connection(s)
I20260811 18:58:38.373631 14326 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-1/data/info.pb
I20260811 18:58:38.378787 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 14326
I20260811 18:58:38.379009 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-1/wal/instance
I20260811 18:58:38.379585 30146 external_mini_cluster.cc:1428] Setting key 9ac1ec0321b7956f7e2607f5f67e957e
I20260811 18:58:38.383049 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.131:0
--local_ip_for_outbound_sockets=127.29.112.131
--webserver_interface=127.29.112.131
--webserver_port=0
--tserver_master_addrs=127.29.112.190:38859
--builtin_ntp_servers=127.29.112.148:43061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260811 18:58:38.388926 14323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:38.376935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.130:54611 (local address 127.29.112.190:38859)
0811 18:58:38.377210 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:38.377217 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:38.378476 (+ 1259us) server_negotiation.cc:407] Connection header received
0811 18:58:38.381004 (+ 2528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:38.381008 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:38.381071 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:38.381156 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:38.382640 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:38.383458 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:38.386027 (+ 2569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:38.386438 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:38.386610 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:38.388203 (+ 1593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:58:38.388208 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:58:38.388212 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:58:38.388229 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:58:38.388260 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:58:38.388272 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:58:38.388342 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:38.388603 (+ 261us) server_negotiation.cc:299] Negotiation successful
0811 18:58:38.388681 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
I20260811 18:58:38.390374 14528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:38.377231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:38859 (local address 127.29.112.130:54611)
0811 18:58:38.378208 (+ 977us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:38.378320 (+ 112us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:38.380701 (+ 2381us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:38.381524 (+ 823us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:38.381535 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:38.381648 (+ 113us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:58:38.382430 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:38.382455 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:38.383624 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:38.383633 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:38.385859 (+ 2226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:38.385877 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:38.386695 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:38.386704 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:38.387038 (+ 334us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:38.387826 (+ 788us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:58:38.387831 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:58:38.387886 (+ 55us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:58:38.387984 (+ 98us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:58:38.388371 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:58:38.388384 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:58:38.388394 (+ 10us) client_negotiation.cc:769] Sending connection context
0811 18:58:38.388571 (+ 177us) client_negotiation.cc:240] Negotiation successful
0811 18:58:38.388982 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":319,"threads_started":1}
I20260811 18:58:38.392882 14526 heartbeater.cc:347] Connected to a master server at 127.29.112.190:38859
I20260811 18:58:38.393253 14526 heartbeater.cc:464] Registering TS with master...
I20260811 18:58:38.394111 14526 heartbeater.cc:511] Master 127.29.112.190:38859 requested a full tablet report, sending...
I20260811 18:58:38.396337 14010 ts_manager.cc:194] Registered new tserver with Master: 00dcb614480749cc8cfa6f79f88aa74d (127.29.112.130:35485)
I20260811 18:58:38.397207 14010 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.112.130:54611
W20260811 18:58:38.753183 14529 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:58:38.753497 14529 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:58:38.753638 14529 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:58:38.761152 14529 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:58:38.761250 14529 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:58:38.761463 14529 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.131
I20260811 18:58:38.770961 14529 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:43061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.112.131
--webserver_port=0
--tserver_master_addrs=127.29.112.190:38859
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.131
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:58:38.772274 14529 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:58:38.774073 14529 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:58:38.787541 14539 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:38.787727 14538 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:38.787624 14541 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:58:38.789037 14529 server_base.cc:1034] running on GCE node
I20260811 18:58:38.790074 14529 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:58:38.791312 14529 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:58:38.792547 14529 hybrid_clock.cc:648] HybridClock initialized: now 1786474718792469 us; error 53 us; skew 500 ppm
I20260811 18:58:38.792886 14529 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 18:58:38.795435 14529 webserver.cc:460] Webserver started at http://127.29.112.131:44889/ using document root <none> and password file <none>
I20260811 18:58:38.796173 14529 fs_manager.cc:359] Metadata directory not provided
I20260811 18:58:38.796294 14529 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:58:38.796656 14529 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:58:38.799646 14529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-2/data/instance:
uuid: "9ebe28be05c64df28687e5bf1c956cf6"
format_stamp: "Formatted at 2026-08-11 18:58:38 on dist-test-slave-h6n0"
server_key: "3f9263afbfc13d51c8e76b7d5f2356a2"
server_key_iv: "ed24a0fff13279a7c3138c3e7a86b6fb"
server_key_version: "encryptionkey@0"
I20260811 18:58:38.800473 14529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-2/wal/instance:
uuid: "9ebe28be05c64df28687e5bf1c956cf6"
format_stamp: "Formatted at 2026-08-11 18:58:38 on dist-test-slave-h6n0"
server_key: "3f9263afbfc13d51c8e76b7d5f2356a2"
server_key_iv: "ed24a0fff13279a7c3138c3e7a86b6fb"
server_key_version: "encryptionkey@0"
I20260811 18:58:38.805925 14529 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 18:58:38.809620 14549 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:38.811100 14529 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:58:38.811291 14529 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-2/wal
uuid: "9ebe28be05c64df28687e5bf1c956cf6"
format_stamp: "Formatted at 2026-08-11 18:58:38 on dist-test-slave-h6n0"
server_key: "3f9263afbfc13d51c8e76b7d5f2356a2"
server_key_iv: "ed24a0fff13279a7c3138c3e7a86b6fb"
server_key_version: "encryptionkey@0"
I20260811 18:58:38.811534 14529 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:58:38.833425 14529 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:58:38.834293 14529 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:58:38.834671 14529 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:58:38.835772 14529 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:58:38.837363 14529 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:58:38.837500 14529 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:38.837628 14529 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:58:38.837723 14529 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:38.878623 14529 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.131:34489
I20260811 18:58:38.878710 14731 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.131:34489 every 8 connection(s)
I20260811 18:58:38.880331 14529 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-2/data/info.pb
I20260811 18:58:38.889722 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 14529
I20260811 18:58:38.889956 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-2/wal/instance
I20260811 18:58:38.890763 30146 external_mini_cluster.cc:1428] Setting key 15b8498595eb177be2cd415775097c88
I20260811 18:58:38.894443 14323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:38.883357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.131:37439 (local address 127.29.112.190:38859)
0811 18:58:38.883559 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:38.883564 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:38.884978 (+ 1414us) server_negotiation.cc:407] Connection header received
0811 18:58:38.886983 (+ 2005us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:38.886991 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:38.887088 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:38.887211 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:38.888364 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:38.889167 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:38.891228 (+ 2061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:38.891807 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:38.891952 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:38.893527 (+ 1575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:58:38.893533 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:58:38.893536 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:58:38.893553 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:58:38.893584 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:58:38.893596 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:58:38.893697 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:38.894024 (+ 327us) server_negotiation.cc:299] Negotiation successful
0811 18:58:38.894137 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 18:58:38.895347 14734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:38.883845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:38859 (local address 127.29.112.131:37439)
0811 18:58:38.884800 (+ 955us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:38.884852 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:38.886684 (+ 1832us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:38.887463 (+ 779us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:38.887472 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:38.887576 (+ 104us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:58:38.888177 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:38.888197 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:38.889367 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:38.889373 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:38.891068 (+ 1695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:38.891081 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:38.892043 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:38.892051 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:38.892358 (+ 307us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:38.893182 (+ 824us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:58:38.893185 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:58:38.893229 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:58:38.893310 (+ 81us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:58:38.893696 (+ 386us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:58:38.893709 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:58:38.893719 (+ 10us) client_negotiation.cc:769] Sending connection context
0811 18:58:38.893939 (+ 220us) client_negotiation.cc:240] Negotiation successful
0811 18:58:38.894248 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":543,"thread_start_us":316,"threads_started":1}
I20260811 18:58:38.896720 14732 heartbeater.cc:347] Connected to a master server at 127.29.112.190:38859
I20260811 18:58:38.897044 14732 heartbeater.cc:464] Registering TS with master...
I20260811 18:58:38.897769 14732 heartbeater.cc:511] Master 127.29.112.190:38859 requested a full tablet report, sending...
I20260811 18:58:38.899094 14321 heartbeater.cc:503] Master 127.29.112.190:38859 was elected leader, sending a full tablet report...
I20260811 18:58:38.899386 14010 ts_manager.cc:194] Registered new tserver with Master: 9ebe28be05c64df28687e5bf1c956cf6 (127.29.112.131:34489)
I20260811 18:58:38.900375 14010 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.112.131:37439
I20260811 18:58:38.907492 30146 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 18:58:38.914183 14323 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:58:38.912560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54648 (local address 127.29.112.190:38859)
0811 18:58:38.912774 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:38.912779 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:38.913311 (+ 532us) server_negotiation.cc:407] Connection header received
0811 18:58:38.913458 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:38.913461 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:38.913540 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:38.913627 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:38.913987 (+ 360us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54648: BlockingRecv error: recv got EOF from 127.0.0.1:54648 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
I20260811 18:58:38.915189 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 14099
I20260811 18:58:38.923650 14743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:38.916366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60038 (local address 127.29.112.129:33023)
0811 18:58:38.916984 (+ 618us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:38.916991 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:38.917018 (+ 27us) server_negotiation.cc:407] Connection header received
0811 18:58:38.917081 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:38.917088 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:38.917292 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:38.917440 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:38.917878 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:38.919109 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:38.920329 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:38.920953 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:38.921063 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:38.922138 (+ 1075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:58:38.922146 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:58:38.922158 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:58:38.922775 (+ 617us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:58:38.922820 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:58:38.922846 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:58:38.922941 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:38.923138 (+ 197us) server_negotiation.cc:299] Negotiation successful
0811 18:58:38.923282 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":460,"thread_start_us":321,"threads_started":1}
I20260811 18:58:38.924935 14315 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:58:39.010193 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 14099
I20260811 18:58:39.033682 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 14326
I20260811 18:58:39.041630 14748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:39.034757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52130 (local address 127.29.112.130:35485)
0811 18:58:39.035277 (+ 520us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:39.035285 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:39.035311 (+ 26us) server_negotiation.cc:407] Connection header received
0811 18:58:39.035378 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:39.035384 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:39.035619 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:39.035766 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:39.036165 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:39.037174 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:39.038379 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:39.039105 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:39.039205 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:39.040250 (+ 1045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:58:39.040258 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:58:39.040270 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:58:39.040869 (+ 599us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:58:39.040914 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:58:39.040944 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:58:39.041033 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:39.041192 (+ 159us) server_negotiation.cc:299] Negotiation successful
0811 18:58:39.041306 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":264,"threads_started":1}
I20260811 18:58:39.042902 14517 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:58:39.128736 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 14326
I20260811 18:58:39.156468 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 14529
I20260811 18:58:39.167384 14753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:39.157647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33152 (local address 127.29.112.131:34489)
0811 18:58:39.158686 (+ 1039us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:39.158696 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:39.158743 (+ 47us) server_negotiation.cc:407] Connection header received
0811 18:58:39.158836 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:39.158844 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:39.159143 (+ 299us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:39.159407 (+ 264us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:39.160252 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:39.161427 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:39.162807 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:39.163605 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:39.163757 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:39.165467 (+ 1710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:58:39.165476 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:58:39.165500 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:58:39.166381 (+ 881us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:58:39.166438 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:58:39.166462 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:58:39.166702 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:39.166877 (+ 175us) server_negotiation.cc:299] Negotiation successful
0811 18:58:39.167007 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":735,"thread_start_us":466,"threads_started":1}
I20260811 18:58:39.168869 14725 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:58:39.254776 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 14529
I20260811 18:58:39.280665 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 13960
I20260811 18:58:39.281802 14057 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:58:39.385747 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 13960
2026-08-11T18:58:39Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2608 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-11T18:58:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:58:39Z Disabled control of system clock
I20260811 18:58:39.478412 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474615205622-30146-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474615205622-30146-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474615205622-30146-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.190:39635
--webserver_interface=127.29.112.190
--webserver_port=0
--builtin_ntp_servers=127.29.112.148:35555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.190:39635
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 18:58:39.820106 14766 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:58:39.820478 14766 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:58:39.820653 14766 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:58:39.827770 14766 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:58:39.827919 14766 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:58:39.828006 14766 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:58:39.828146 14766 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:58:39.828230 14766 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:58:39.837095 14766 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:35555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474615205622-30146-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474615205622-30146-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.112.190:39635
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.190:39635
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.112.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474615205622-30146-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:58:39.838335 14766 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:58:39.840104 14766 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:58:39.852264 14781 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:39.852206 14775 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:39.852447 14774 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:58:39.853554 14766 server_base.cc:1034] running on GCE node
I20260811 18:58:39.854458 14766 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:58:39.855989 14766 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:58:39.857223 14766 hybrid_clock.cc:648] HybridClock initialized: now 1786474719857180 us; error 30 us; skew 500 ppm
I20260811 18:58:39.857728 14766 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 18:58:39.860126 14766 webserver.cc:460] Webserver started at http://127.29.112.190:36677/ using document root <none> and password file <none>
I20260811 18:58:39.860821 14766 fs_manager.cc:359] Metadata directory not provided
I20260811 18:58:39.860937 14766 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:58:39.861292 14766 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:58:39.864208 14766 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474615205622-30146-0/minicluster-data/master-0/data/instance:
uuid: "1a8af681233c40d5b4c861078e7aa676"
format_stamp: "Formatted at 2026-08-11 18:58:39 on dist-test-slave-h6n0"
server_key: "45bf48f2c33e65076dbf3ba8947ccdf9"
server_key_iv: "b7750b9de6ac055ccef1389e3bfa9cce"
server_key_version: "encryptionkey@0"
I20260811 18:58:39.864981 14766 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474615205622-30146-0/minicluster-data/master-0/wal/instance:
uuid: "1a8af681233c40d5b4c861078e7aa676"
format_stamp: "Formatted at 2026-08-11 18:58:39 on dist-test-slave-h6n0"
server_key: "45bf48f2c33e65076dbf3ba8947ccdf9"
server_key_iv: "b7750b9de6ac055ccef1389e3bfa9cce"
server_key_version: "encryptionkey@0"
I20260811 18:58:39.870075 14766 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260811 18:58:39.873827 14790 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:39.875327 14766 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 18:58:39.875536 14766 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474615205622-30146-0/minicluster-data/master-0/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474615205622-30146-0/minicluster-data/master-0/wal
uuid: "1a8af681233c40d5b4c861078e7aa676"
format_stamp: "Formatted at 2026-08-11 18:58:39 on dist-test-slave-h6n0"
server_key: "45bf48f2c33e65076dbf3ba8947ccdf9"
server_key_iv: "b7750b9de6ac055ccef1389e3bfa9cce"
server_key_version: "encryptionkey@0"
I20260811 18:58:39.875764 14766 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474615205622-30146-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474615205622-30146-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474615205622-30146-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:58:39.881686 14766 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:58:39.882450 14766 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:58:39.882771 14766 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:58:39.902127 14766 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.190:39635
I20260811 18:58:39.902174 14873 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.190:39635 every 8 connection(s)
I20260811 18:58:39.903558 14766 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
I20260811 18:58:39.907210 14874 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:58:39.912195 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 14766
I20260811 18:58:39.912405 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474615205622-30146-0/minicluster-data/master-0/wal/instance
I20260811 18:58:39.913237 30146 external_mini_cluster.cc:1428] Setting key 6f9562d8e9144f2d47951182be56e7d3
I20260811 18:58:39.918774 14874 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a8af681233c40d5b4c861078e7aa676: Bootstrap starting.
I20260811 18:58:39.922489 14874 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1a8af681233c40d5b4c861078e7aa676: Neither blocks nor log segments found. Creating new log.
I20260811 18:58:39.922586 14877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:39.915615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42402 (local address 127.29.112.190:39635)
0811 18:58:39.916677 (+ 1062us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:39.916688 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:39.916728 (+ 40us) server_negotiation.cc:407] Connection header received
0811 18:58:39.918333 (+ 1605us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:39.918355 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:39.918795 (+ 440us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:39.919161 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:39.920309 (+ 1148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:58:39.920315 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:58:39.920330 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:58:39.920344 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:58:39.920410 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:58:39.920427 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:58:39.920597 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:39.920719 (+ 122us) server_negotiation.cc:299] Negotiation successful
0811 18:58:39.921362 (+ 643us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":621,"thread_start_us":402,"threads_started":1}
I20260811 18:58:39.923651 14874 log.cc:824] T 00000000000000000000000000000000 P 1a8af681233c40d5b4c861078e7aa676: Log is configured to *not* fsync() on all Append() calls
I20260811 18:58:39.926725 14874 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a8af681233c40d5b4c861078e7aa676: No bootstrap required, opened a new log
I20260811 18:58:39.937546 14874 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1a8af681233c40d5b4c861078e7aa676 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a8af681233c40d5b4c861078e7aa676" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 39635 } }
I20260811 18:58:39.937932 14874 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1a8af681233c40d5b4c861078e7aa676 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:58:39.938019 14874 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1a8af681233c40d5b4c861078e7aa676 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a8af681233c40d5b4c861078e7aa676, State: Initialized, Role: FOLLOWER
I20260811 18:58:39.938652 14874 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1a8af681233c40d5b4c861078e7aa676 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a8af681233c40d5b4c861078e7aa676" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 39635 } }
I20260811 18:58:39.938902 14874 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1a8af681233c40d5b4c861078e7aa676 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:58:39.939018 14874 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1a8af681233c40d5b4c861078e7aa676 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:58:39.939155 14874 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1a8af681233c40d5b4c861078e7aa676 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:58:39.942293 14874 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1a8af681233c40d5b4c861078e7aa676 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a8af681233c40d5b4c861078e7aa676" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 39635 } }
I20260811 18:58:39.942849 14874 leader_election.cc:302] T 00000000000000000000000000000000 P 1a8af681233c40d5b4c861078e7aa676 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1a8af681233c40d5b4c861078e7aa676; no voters:
I20260811 18:58:39.943442 14874 leader_election.cc:288] T 00000000000000000000000000000000 P 1a8af681233c40d5b4c861078e7aa676 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:58:39.943588 14881 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1a8af681233c40d5b4c861078e7aa676 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:58:39.944348 14881 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1a8af681233c40d5b4c861078e7aa676 [term 1 LEADER]: Becoming Leader. State: Replica: 1a8af681233c40d5b4c861078e7aa676, State: Running, Role: LEADER
I20260811 18:58:39.944837 14881 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1a8af681233c40d5b4c861078e7aa676 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a8af681233c40d5b4c861078e7aa676" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 39635 } }
I20260811 18:58:39.945523 14874 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1a8af681233c40d5b4c861078e7aa676 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:58:39.947687 14885 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a8af681233c40d5b4c861078e7aa676 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a8af681233c40d5b4c861078e7aa676. Latest consensus state: current_term: 1 leader_uuid: "1a8af681233c40d5b4c861078e7aa676" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a8af681233c40d5b4c861078e7aa676" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 39635 } } }
I20260811 18:58:39.948139 14885 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a8af681233c40d5b4c861078e7aa676 [sys.catalog]: This master's current role is: LEADER
I20260811 18:58:39.949028 14883 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a8af681233c40d5b4c861078e7aa676 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1a8af681233c40d5b4c861078e7aa676" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a8af681233c40d5b4c861078e7aa676" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 39635 } } }
I20260811 18:58:39.949450 14883 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a8af681233c40d5b4c861078e7aa676 [sys.catalog]: This master's current role is: LEADER
I20260811 18:58:39.950109 14888 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:58:39.955590 14888 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:58:39.965197 14888 catalog_manager.cc:1347] Generated new cluster ID: 779b8b852f134b3b83c52770a21ccace
I20260811 18:58:39.965613 14888 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:58:39.991839 14888 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:58:39.992264 14888 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:58:40.001263 14888 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1a8af681233c40d5b4c861078e7aa676: Generated new TSK 0
I20260811 18:58:40.002362 14888 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:58:40.015725 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.129:0
--local_ip_for_outbound_sockets=127.29.112.129
--webserver_interface=127.29.112.129
--webserver_port=0
--tserver_master_addrs=127.29.112.190:39635
--builtin_ntp_servers=127.29.112.148:35555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 18:58:40.402573 14905 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:58:40.402966 14905 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:58:40.403168 14905 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:58:40.411785 14905 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:58:40.411965 14905 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:58:40.412161 14905 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.129
I20260811 18:58:40.422894 14905 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:35555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.112.129
--webserver_port=0
--tserver_master_addrs=127.29.112.190:39635
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.129
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:58:40.424906 14905 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:58:40.427747 14905 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:58:40.443506 14919 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:40.443583 14921 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:40.443534 14916 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:58:40.444449 14905 server_base.cc:1034] running on GCE node
I20260811 18:58:40.445123 14905 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:58:40.446338 14905 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:58:40.447587 14905 hybrid_clock.cc:648] HybridClock initialized: now 1786474720447545 us; error 36 us; skew 500 ppm
I20260811 18:58:40.447932 14905 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 18:58:40.454869 14905 webserver.cc:460] Webserver started at http://127.29.112.129:45297/ using document root <none> and password file <none>
I20260811 18:58:40.455655 14905 fs_manager.cc:359] Metadata directory not provided
I20260811 18:58:40.455757 14905 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:58:40.456244 14905 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:58:40.459329 14905 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-0/data/instance:
uuid: "d5af5fb458e4465e81a89a0e5e7b8e01"
format_stamp: "Formatted at 2026-08-11 18:58:40 on dist-test-slave-h6n0"
server_key: "002203e3fefc0021d3bcb24fa440a3cb"
server_key_iv: "f00e06337b2a0ed0b3caba8d1d9411b2"
server_key_version: "encryptionkey@0"
I20260811 18:58:40.460178 14905 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-0/wal/instance:
uuid: "d5af5fb458e4465e81a89a0e5e7b8e01"
format_stamp: "Formatted at 2026-08-11 18:58:40 on dist-test-slave-h6n0"
server_key: "002203e3fefc0021d3bcb24fa440a3cb"
server_key_iv: "f00e06337b2a0ed0b3caba8d1d9411b2"
server_key_version: "encryptionkey@0"
I20260811 18:58:40.466027 14905 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 18:58:40.469740 14931 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:40.471411 14905 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:58:40.471563 14905 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-0/wal
uuid: "d5af5fb458e4465e81a89a0e5e7b8e01"
format_stamp: "Formatted at 2026-08-11 18:58:40 on dist-test-slave-h6n0"
server_key: "002203e3fefc0021d3bcb24fa440a3cb"
server_key_iv: "f00e06337b2a0ed0b3caba8d1d9411b2"
server_key_version: "encryptionkey@0"
I20260811 18:58:40.471751 14905 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:58:40.477658 14905 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:58:40.478479 14905 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:58:40.478852 14905 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:58:40.479921 14905 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:58:40.481443 14905 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:58:40.481564 14905 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:40.481696 14905 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:58:40.481782 14905 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:40.522099 14905 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.129:43099
I20260811 18:58:40.522163 15112 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.129:43099 every 8 connection(s)
I20260811 18:58:40.523705 14905 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-0/data/info.pb
I20260811 18:58:40.532342 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 14905
I20260811 18:58:40.532567 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-0/wal/instance
I20260811 18:58:40.533035 15117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:40.526767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.129:60211 (local address 127.29.112.190:39635)
0811 18:58:40.527289 (+ 522us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:40.527293 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:40.528476 (+ 1183us) server_negotiation.cc:407] Connection header received
0811 18:58:40.530220 (+ 1744us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:40.530225 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:40.530304 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:40.530428 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:40.531947 (+ 1519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:58:40.531952 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:58:40.531955 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:58:40.531972 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:58:40.532021 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:58:40.532033 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:58:40.532147 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:40.532569 (+ 422us) server_negotiation.cc:299] Negotiation successful
0811 18:58:40.532716 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":278,"threads_started":1}
I20260811 18:58:40.533435 30146 external_mini_cluster.cc:1428] Setting key 2a0829c9d4d62a0bf99698658e6a89e1
I20260811 18:58:40.534226 15118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:40.527385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:39635 (local address 127.29.112.129:60211)
0811 18:58:40.528306 (+ 921us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:40.528344 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:40.529949 (+ 1605us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:40.530738 (+ 789us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:40.530746 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:40.530835 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:58:40.531573 (+ 738us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:58:40.531576 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:58:40.531615 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:58:40.531703 (+ 88us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:58:40.532157 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:58:40.532171 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:58:40.532180 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 18:58:40.532510 (+ 330us) client_negotiation.cc:240] Negotiation successful
0811 18:58:40.532802 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":278,"threads_started":1}
I20260811 18:58:40.535908 15115 heartbeater.cc:347] Connected to a master server at 127.29.112.190:39635
I20260811 18:58:40.536228 15115 heartbeater.cc:464] Registering TS with master...
I20260811 18:58:40.536671 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.130:0
--local_ip_for_outbound_sockets=127.29.112.130
--webserver_interface=127.29.112.130
--webserver_port=0
--tserver_master_addrs=127.29.112.190:39635
--builtin_ntp_servers=127.29.112.148:35555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260811 18:58:40.536898 15115 heartbeater.cc:511] Master 127.29.112.190:39635 requested a full tablet report, sending...
I20260811 18:58:40.539067 14821 ts_manager.cc:194] Registered new tserver with Master: d5af5fb458e4465e81a89a0e5e7b8e01 (127.29.112.129:43099)
W20260811 18:58:40.884472 15120 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:58:40.884744 15120 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:58:40.884886 15120 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:58:40.892036 15120 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:58:40.892136 15120 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:58:40.892305 15120 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.130
I20260811 18:58:40.901517 15120 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:35555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.112.130
--webserver_port=0
--tserver_master_addrs=127.29.112.190:39635
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.130
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:58:40.902865 15120 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:58:40.904633 15120 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:58:40.919934 15125 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:40.919804 15130 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:40.920078 15127 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:58:40.921129 15120 server_base.cc:1034] running on GCE node
I20260811 18:58:40.921855 15120 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:58:40.922930 15120 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:58:40.924196 15120 hybrid_clock.cc:648] HybridClock initialized: now 1786474720924153 us; error 57 us; skew 500 ppm
I20260811 18:58:40.924556 15120 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 18:58:40.929590 15120 webserver.cc:460] Webserver started at http://127.29.112.130:33011/ using document root <none> and password file <none>
I20260811 18:58:40.930289 15120 fs_manager.cc:359] Metadata directory not provided
I20260811 18:58:40.930439 15120 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:58:40.930787 15120 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:58:40.933620 15120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-1/data/instance:
uuid: "6fa051d9b2914cadbabea5e2aa29b234"
format_stamp: "Formatted at 2026-08-11 18:58:40 on dist-test-slave-h6n0"
server_key: "4591bacd4dd9abcfe42c112b4321c5af"
server_key_iv: "bae6ccee504996d4dbe9bfafd266e80e"
server_key_version: "encryptionkey@0"
I20260811 18:58:40.934429 15120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-1/wal/instance:
uuid: "6fa051d9b2914cadbabea5e2aa29b234"
format_stamp: "Formatted at 2026-08-11 18:58:40 on dist-test-slave-h6n0"
server_key: "4591bacd4dd9abcfe42c112b4321c5af"
server_key_iv: "bae6ccee504996d4dbe9bfafd266e80e"
server_key_version: "encryptionkey@0"
I20260811 18:58:40.939930 15120 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 18:58:40.943604 15140 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:40.945171 15120 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 18:58:40.945381 15120 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-1/wal
uuid: "6fa051d9b2914cadbabea5e2aa29b234"
format_stamp: "Formatted at 2026-08-11 18:58:40 on dist-test-slave-h6n0"
server_key: "4591bacd4dd9abcfe42c112b4321c5af"
server_key_iv: "bae6ccee504996d4dbe9bfafd266e80e"
server_key_version: "encryptionkey@0"
I20260811 18:58:40.945621 15120 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:58:40.951367 15120 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:58:40.952135 15120 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:58:40.952410 15120 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:58:40.953298 15120 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:58:40.954779 15120 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:58:40.954923 15120 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:40.955063 15120 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:58:40.955155 15120 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:40.991225 15120 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.130:38499
I20260811 18:58:40.991278 15315 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.130:38499 every 8 connection(s)
I20260811 18:58:40.992749 15120 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-1/data/info.pb
I20260811 18:58:40.998935 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 15120
I20260811 18:58:40.999118 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-1/wal/instance
I20260811 18:58:40.999646 30146 external_mini_cluster.cc:1428] Setting key 6fbb90e767f381e5ce063b01690bef85
I20260811 18:58:41.000937 15117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:40.995502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.130:53795 (local address 127.29.112.190:39635)
0811 18:58:40.995697 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:40.995702 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:40.996904 (+ 1202us) server_negotiation.cc:407] Connection header received
0811 18:58:40.998633 (+ 1729us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:40.998639 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:40.998713 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:40.998827 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:41.000247 (+ 1420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:58:41.000251 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:58:41.000255 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:58:41.000270 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:58:41.000303 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:58:41.000315 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:58:41.000483 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:41.000645 (+ 162us) server_negotiation.cc:299] Negotiation successful
0811 18:58:41.000713 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 18:58:41.002007 15319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:40.995869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:39635 (local address 127.29.112.130:53795)
0811 18:58:40.996752 (+ 883us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:40.996787 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:40.998314 (+ 1527us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:40.999076 (+ 762us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:40.999089 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:40.999205 (+ 116us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:58:40.999942 (+ 737us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:58:40.999945 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:58:40.999980 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:58:41.000057 (+ 77us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:58:41.000451 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:58:41.000460 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:58:41.000465 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 18:58:41.000701 (+ 236us) client_negotiation.cc:240] Negotiation successful
0811 18:58:41.000933 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":273,"threads_started":1}
I20260811 18:58:41.002844 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.131:0
--local_ip_for_outbound_sockets=127.29.112.131
--webserver_interface=127.29.112.131
--webserver_port=0
--tserver_master_addrs=127.29.112.190:39635
--builtin_ntp_servers=127.29.112.148:35555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260811 18:58:41.003495 15316 heartbeater.cc:347] Connected to a master server at 127.29.112.190:39635
I20260811 18:58:41.003804 15316 heartbeater.cc:464] Registering TS with master...
I20260811 18:58:41.004477 15316 heartbeater.cc:511] Master 127.29.112.190:39635 requested a full tablet report, sending...
I20260811 18:58:41.005935 14821 ts_manager.cc:194] Registered new tserver with Master: 6fa051d9b2914cadbabea5e2aa29b234 (127.29.112.130:38499)
W20260811 18:58:41.366892 15321 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:58:41.367183 15321 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:58:41.367349 15321 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:58:41.378322 15321 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:58:41.378491 15321 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:58:41.378738 15321 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.131
I20260811 18:58:41.392765 15321 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:35555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.112.131
--webserver_port=0
--tserver_master_addrs=127.29.112.190:39635
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.131
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:58:41.394196 15321 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:58:41.396576 15321 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:58:41.410298 15328 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:41.410405 15334 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:41.410405 15329 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:58:41.412303 15321 server_base.cc:1034] running on GCE node
I20260811 18:58:41.413230 15321 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:58:41.414345 15321 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:58:41.415947 15321 hybrid_clock.cc:648] HybridClock initialized: now 1786474721415707 us; error 80 us; skew 500 ppm
I20260811 18:58:41.416347 15321 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 18:58:41.419059 15321 webserver.cc:460] Webserver started at http://127.29.112.131:40745/ using document root <none> and password file <none>
I20260811 18:58:41.419811 15321 fs_manager.cc:359] Metadata directory not provided
I20260811 18:58:41.419960 15321 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:58:41.420327 15321 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:58:41.423410 15321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-2/data/instance:
uuid: "16f2565ef3ea4332802472c510a4f5d0"
format_stamp: "Formatted at 2026-08-11 18:58:41 on dist-test-slave-h6n0"
server_key: "22158e9f3a62b3b362fcd84a61841f9a"
server_key_iv: "9111d10cdc3e46f90123f728508fbd89"
server_key_version: "encryptionkey@0"
I20260811 18:58:41.424290 15321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-2/wal/instance:
uuid: "16f2565ef3ea4332802472c510a4f5d0"
format_stamp: "Formatted at 2026-08-11 18:58:41 on dist-test-slave-h6n0"
server_key: "22158e9f3a62b3b362fcd84a61841f9a"
server_key_iv: "9111d10cdc3e46f90123f728508fbd89"
server_key_version: "encryptionkey@0"
I20260811 18:58:41.429842 15321 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 18:58:41.433537 15344 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:41.434999 15321 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 18:58:41.435182 15321 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-2/wal
uuid: "16f2565ef3ea4332802472c510a4f5d0"
format_stamp: "Formatted at 2026-08-11 18:58:41 on dist-test-slave-h6n0"
server_key: "22158e9f3a62b3b362fcd84a61841f9a"
server_key_iv: "9111d10cdc3e46f90123f728508fbd89"
server_key_version: "encryptionkey@0"
I20260811 18:58:41.435418 15321 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:58:41.442631 15321 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:58:41.443578 15321 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:58:41.443900 15321 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:58:41.444931 15321 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:58:41.446796 15321 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:58:41.446933 15321 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:41.447072 15321 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:58:41.447172 15321 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:41.490322 15321 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.131:33757
I20260811 18:58:41.490372 15519 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.131:33757 every 8 connection(s)
I20260811 18:58:41.491703 15321 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-2/data/info.pb
I20260811 18:58:41.497005 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 15321
I20260811 18:58:41.497190 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474615205622-30146-0/minicluster-data/ts-2/wal/instance
I20260811 18:58:41.497787 30146 external_mini_cluster.cc:1428] Setting key 083fa4b51048999948d6f2604bae35b0
I20260811 18:58:41.501906 15117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:41.494470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.131:41571 (local address 127.29.112.190:39635)
0811 18:58:41.494715 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:41.494720 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:41.496174 (+ 1454us) server_negotiation.cc:407] Connection header received
0811 18:58:41.498783 (+ 2609us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:41.498788 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:41.498897 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:41.499020 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:41.500880 (+ 1860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:58:41.500885 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:58:41.500888 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:58:41.500903 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:58:41.500943 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:58:41.500956 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:58:41.501102 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:41.501455 (+ 353us) server_negotiation.cc:299] Negotiation successful
0811 18:58:41.501550 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112}
I20260811 18:58:41.502691 15522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:41.494935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:39635 (local address 127.29.112.131:41571)
0811 18:58:41.495853 (+ 918us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:41.495906 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:41.498434 (+ 2528us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:41.499360 (+ 926us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:41.499373 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:41.499495 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:58:41.500570 (+ 1075us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:58:41.500573 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:58:41.500610 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:58:41.500692 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:58:41.501118 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:58:41.501127 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:58:41.501134 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 18:58:41.501373 (+ 239us) client_negotiation.cc:240] Negotiation successful
0811 18:58:41.501581 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":300,"threads_started":1}
I20260811 18:58:41.504127 15520 heartbeater.cc:347] Connected to a master server at 127.29.112.190:39635
I20260811 18:58:41.504429 15520 heartbeater.cc:464] Registering TS with master...
I20260811 18:58:41.505023 15520 heartbeater.cc:511] Master 127.29.112.190:39635 requested a full tablet report, sending...
I20260811 18:58:41.506466 14821 ts_manager.cc:194] Registered new tserver with Master: 16f2565ef3ea4332802472c510a4f5d0 (127.29.112.131:33757)
I20260811 18:58:41.513365 30146 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 18:58:41.519770 15117 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:58:41.518092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42404 (local address 127.29.112.190:39635)
0811 18:58:41.518273 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:41.518278 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:41.518832 (+ 554us) server_negotiation.cc:407] Connection header received
0811 18:58:41.518960 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:41.518963 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:41.519012 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:41.519108 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:41.519467 (+ 359us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42404: BlockingRecv error: recv got EOF from 127.0.0.1:42404 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 18:58:41.520466 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 14905
I20260811 18:58:41.525249 15531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:41.521421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44896 (local address 127.29.112.129:43099)
0811 18:58:41.521930 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:41.521939 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:41.521968 (+ 29us) server_negotiation.cc:407] Connection header received
0811 18:58:41.522057 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:41.522065 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:41.522301 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:41.522473 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:41.524006 (+ 1533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:58:41.524015 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:58:41.524029 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:58:41.524053 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:58:41.524115 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:58:41.524138 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:58:41.524363 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:41.524499 (+ 136us) server_negotiation.cc:299] Negotiation successful
0811 18:58:41.524755 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":221,"threads_started":1}
I20260811 18:58:41.527220 15107 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:58:41.609757 15115 heartbeater.cc:503] Master 127.29.112.190:39635 was elected leader, sending a full tablet report...
I20260811 18:58:41.611039 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 14905
W20260811 18:58:41.635016 14796 connection.cc:535] server connection from 127.29.112.129:60211 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 18:58:41.635398 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 15120
I20260811 18:58:41.639529 15535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:41.636473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54422 (local address 127.29.112.130:38499)
0811 18:58:41.637070 (+ 597us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:41.637085 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:41.637112 (+ 27us) server_negotiation.cc:407] Connection header received
0811 18:58:41.637176 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:41.637182 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:41.637412 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:41.637534 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:41.638641 (+ 1107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:58:41.638648 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:58:41.638659 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:58:41.638681 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:58:41.638726 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:58:41.638741 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:58:41.638847 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:41.639195 (+ 348us) server_negotiation.cc:299] Negotiation successful
0811 18:58:41.639284 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":449,"thread_start_us":309,"threads_started":1}
I20260811 18:58:41.640813 15311 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:58:41.722771 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 15120
I20260811 18:58:41.748713 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 15321
I20260811 18:58:41.752914 15537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:41.749653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35010 (local address 127.29.112.131:33757)
0811 18:58:41.750314 (+ 661us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:41.750321 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:41.750353 (+ 32us) server_negotiation.cc:407] Connection header received
0811 18:58:41.750422 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:41.750428 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:41.750687 (+ 259us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:41.750871 (+ 184us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:41.752123 (+ 1252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:58:41.752133 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:58:41.752148 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:58:41.752171 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:58:41.752215 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:58:41.752230 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:58:41.752422 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:41.752505 (+ 83us) server_negotiation.cc:299] Negotiation successful
0811 18:58:41.752609 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":511,"thread_start_us":341,"threads_started":1}
I20260811 18:58:41.754263 15511 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:58:41.838025 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 15321
I20260811 18:58:41.863912 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 14766
I20260811 18:58:41.865257 14867 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:58:41.972136 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 14766
2026-08-11T18:58:42Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2589 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:58:42 dist-test-slave-h6n0 krb5kdc[15543](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:58:42 dist-test-slave-h6n0 krb5kdc[15543](info): set up 2 sockets
Aug 11 18:58:42 dist-test-slave-h6n0 krb5kdc[15543](info): commencing operation
krb5kdc: starting...
W20260811 18:58:43.390094 30146 mini_kdc.cc:121] Time spent starting KDC: real 1.362s user 0.000s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/krb5kdc/test-user.keytab.
Aug 11 18:58:43 dist-test-slave-h6n0 krb5kdc[15543](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474723, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:58:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:58:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.112.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab.
Entry for principal kudu/127.29.112.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab.
I20260811 18:58:43.698151 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.190:34387
--webserver_interface=127.29.112.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab
--principal=kudu/127.29.112.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:38035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.190:34387
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:58:44.044533 15567 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:58:44.044857 15567 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:58:44.045073 15567 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:58:44.052593 15567 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:58:44.052714 15567 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:58:44.052862 15567 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:58:44.052951 15567 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:58:44.053037 15567 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:58:44.063033 15567 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:38035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.112.190:34387
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab
--principal=kudu/127.29.112.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.190:34387
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.112.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:58:44.064385 15567 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:58:44.065688 15567 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:58:44.078394 15579 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:44.078481 15577 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:44.078462 15583 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:58:44.079561 15567 server_base.cc:1034] running on GCE node
I20260811 18:58:44.080497 15567 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:58:44.081800 15567 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:58:44.083038 15567 hybrid_clock.cc:648] HybridClock initialized: now 1786474724083004 us; error 47 us; skew 500 ppm
I20260811 18:58:44.083380 15567 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:58:44 dist-test-slave-h6n0 krb5kdc[15543](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474724, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:58:44.087612 15567 init.cc:377] Logged in from keytab as kudu/127.29.112.190@KRBTEST.COM (short username kudu)
I20260811 18:58:44.089124 15567 webserver.cc:460] Webserver started at http://127.29.112.190:33457/ using document root <none> and password file <none>
I20260811 18:58:44.089793 15567 fs_manager.cc:359] Metadata directory not provided
I20260811 18:58:44.089952 15567 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:58:44.090272 15567 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:58:44.093284 15567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/minicluster-data/master-0/data/instance:
uuid: "0d76b2dbe757456dac0226fe61ed305c"
format_stamp: "Formatted at 2026-08-11 18:58:44 on dist-test-slave-h6n0"
server_key: "777bcc968da45141ac106fcb2b81af2c"
server_key_iv: "9c4d3a6805ed8d6a5874b11d53b4e5c0"
server_key_version: "encryptionkey@0"
I20260811 18:58:44.094051 15567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/minicluster-data/master-0/wal/instance:
uuid: "0d76b2dbe757456dac0226fe61ed305c"
format_stamp: "Formatted at 2026-08-11 18:58:44 on dist-test-slave-h6n0"
server_key: "777bcc968da45141ac106fcb2b81af2c"
server_key_iv: "9c4d3a6805ed8d6a5874b11d53b4e5c0"
server_key_version: "encryptionkey@0"
I20260811 18:58:44.099310 15567 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 18:58:44.102802 15592 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:44.104085 15567 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 18:58:44.104281 15567 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/minicluster-data/master-0/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/minicluster-data/master-0/wal
uuid: "0d76b2dbe757456dac0226fe61ed305c"
format_stamp: "Formatted at 2026-08-11 18:58:44 on dist-test-slave-h6n0"
server_key: "777bcc968da45141ac106fcb2b81af2c"
server_key_iv: "9c4d3a6805ed8d6a5874b11d53b4e5c0"
server_key_version: "encryptionkey@0"
I20260811 18:58:44.104501 15567 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:58:44.118311 15567 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:58:44.122265 15567 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:58:44.122599 15567 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:58:44.143690 15567 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.190:34387
I20260811 18:58:44.143726 15668 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.190:34387 every 8 connection(s)
I20260811 18:58:44.145216 15567 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
I20260811 18:58:44.148643 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 15567
I20260811 18:58:44.148867 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/minicluster-data/master-0/wal/instance
I20260811 18:58:44.149593 15671 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:58:44.149852 30146 external_mini_cluster.cc:1428] Setting key 5d51e6bca78e7b6b863a45e101ab8506
I20260811 18:58:44.166373 15671 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d76b2dbe757456dac0226fe61ed305c: Bootstrap starting.
Aug 11 18:58:44 dist-test-slave-h6n0 krb5kdc[15543](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474723, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 18:58:44.170460 15671 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0d76b2dbe757456dac0226fe61ed305c: Neither blocks nor log segments found. Creating new log.
I20260811 18:58:44.171830 15671 log.cc:824] T 00000000000000000000000000000000 P 0d76b2dbe757456dac0226fe61ed305c: Log is configured to *not* fsync() on all Append() calls
I20260811 18:58:44.175202 15671 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d76b2dbe757456dac0226fe61ed305c: No bootstrap required, opened a new log
I20260811 18:58:44.177397 15679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:44.152507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54548 (local address 127.29.112.190:34387)
0811 18:58:44.153560 (+ 1053us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:44.153571 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:44.153621 (+ 50us) server_negotiation.cc:407] Connection header received
0811 18:58:44.155447 (+ 1826us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:44.155470 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:44.155871 (+ 401us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:44.156221 (+ 350us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:44.158198 (+ 1977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:44.159555 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:44.161464 (+ 1909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:44.162500 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:44.162734 (+ 234us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:44.167930 (+ 5196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:58:44.167947 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:58:44.167969 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:58:44.168039 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:58:44.172193 (+ 4154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:44.173289 (+ 1096us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:44.173301 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:44.173311 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:44.173530 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:44.174073 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:44.174078 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:44.174081 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:44.174835 (+ 754us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:58:44.175152 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:44.175596 (+ 444us) server_negotiation.cc:299] Negotiation successful
0811 18:58:44.176073 (+ 477us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":827,"thread_start_us":566,"threads_started":1}
I20260811 18:58:44.184168 15671 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0d76b2dbe757456dac0226fe61ed305c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d76b2dbe757456dac0226fe61ed305c" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 34387 } }
I20260811 18:58:44.184481 15671 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0d76b2dbe757456dac0226fe61ed305c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:58:44.184552 15671 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0d76b2dbe757456dac0226fe61ed305c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d76b2dbe757456dac0226fe61ed305c, State: Initialized, Role: FOLLOWER
I20260811 18:58:44.185073 15671 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0d76b2dbe757456dac0226fe61ed305c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d76b2dbe757456dac0226fe61ed305c" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 34387 } }
I20260811 18:58:44.185273 15671 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0d76b2dbe757456dac0226fe61ed305c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:58:44.185369 15671 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0d76b2dbe757456dac0226fe61ed305c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:58:44.185484 15671 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0d76b2dbe757456dac0226fe61ed305c [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:58:44.187682 15671 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0d76b2dbe757456dac0226fe61ed305c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d76b2dbe757456dac0226fe61ed305c" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 34387 } }
I20260811 18:58:44.188167 15671 leader_election.cc:302] T 00000000000000000000000000000000 P 0d76b2dbe757456dac0226fe61ed305c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0d76b2dbe757456dac0226fe61ed305c; no voters:
I20260811 18:58:44.188733 15671 leader_election.cc:288] T 00000000000000000000000000000000 P 0d76b2dbe757456dac0226fe61ed305c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:58:44.188890 15686 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0d76b2dbe757456dac0226fe61ed305c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:58:44.189569 15686 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0d76b2dbe757456dac0226fe61ed305c [term 1 LEADER]: Becoming Leader. State: Replica: 0d76b2dbe757456dac0226fe61ed305c, State: Running, Role: LEADER
I20260811 18:58:44.190207 15686 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0d76b2dbe757456dac0226fe61ed305c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d76b2dbe757456dac0226fe61ed305c" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 34387 } }
I20260811 18:58:44.190786 15671 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0d76b2dbe757456dac0226fe61ed305c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:58:44.193167 15688 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d76b2dbe757456dac0226fe61ed305c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0d76b2dbe757456dac0226fe61ed305c. Latest consensus state: current_term: 1 leader_uuid: "0d76b2dbe757456dac0226fe61ed305c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d76b2dbe757456dac0226fe61ed305c" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 34387 } } }
I20260811 18:58:44.193463 15688 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d76b2dbe757456dac0226fe61ed305c [sys.catalog]: This master's current role is: LEADER
I20260811 18:58:44.193941 15687 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d76b2dbe757456dac0226fe61ed305c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0d76b2dbe757456dac0226fe61ed305c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d76b2dbe757456dac0226fe61ed305c" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 34387 } } }
I20260811 18:58:44.194320 15695 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:58:44.194351 15687 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d76b2dbe757456dac0226fe61ed305c [sys.catalog]: This master's current role is: LEADER
I20260811 18:58:44.198629 15695 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:58:44.207562 15695 catalog_manager.cc:1347] Generated new cluster ID: efd8e3e9660c445cb78d46ba127143e6
I20260811 18:58:44.207679 15695 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:58:44.231979 15695 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:58:44.233843 15695 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:58:44.248853 15695 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0d76b2dbe757456dac0226fe61ed305c: Generated new TSK 0
I20260811 18:58:44.249927 15695 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.112.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.129.keytab.
Entry for principal kudu/127.29.112.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.129.keytab.
I20260811 18:58:44.330487 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.129:0
--local_ip_for_outbound_sockets=127.29.112.129
--webserver_interface=127.29.112.129
--webserver_port=0
--tserver_master_addrs=127.29.112.190:34387
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.129.keytab
--principal=kudu/127.29.112.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:38035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:58:44.674206 15721 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:58:44.674558 15721 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:58:44.674660 15721 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:58:44.682372 15721 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:58:44.682479 15721 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:58:44.682651 15721 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.129
I20260811 18:58:44.692528 15721 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:38035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.129.keytab
--principal=kudu/127.29.112.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.112.129
--webserver_port=0
--tserver_master_addrs=127.29.112.190:34387
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.129
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:58:44.693785 15721 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:58:44.695142 15721 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:58:44.708616 15730 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:44.709072 15732 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:44.709311 15736 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:58:44.710255 15721 server_base.cc:1034] running on GCE node
I20260811 18:58:44.711025 15721 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:58:44.712069 15721 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:58:44.713331 15721 hybrid_clock.cc:648] HybridClock initialized: now 1786474724713281 us; error 62 us; skew 500 ppm
I20260811 18:58:44.713688 15721 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:58:44 dist-test-slave-h6n0 krb5kdc[15543](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474724, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:58:44.717381 15721 init.cc:377] Logged in from keytab as kudu/127.29.112.129@KRBTEST.COM (short username kudu)
I20260811 18:58:44.719034 15721 webserver.cc:460] Webserver started at http://127.29.112.129:44101/ using document root <none> and password file <none>
I20260811 18:58:44.719743 15721 fs_manager.cc:359] Metadata directory not provided
I20260811 18:58:44.719918 15721 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:58:44.720259 15721 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:58:44.723421 15721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/minicluster-data/ts-0/data/instance:
uuid: "8d2777c33f3a4f5980f876dba840583c"
format_stamp: "Formatted at 2026-08-11 18:58:44 on dist-test-slave-h6n0"
server_key: "cdd578f600ab0100bb2beb5c06ebc771"
server_key_iv: "9754ea3fcc07fa19ccd2d92ac94c33c5"
server_key_version: "encryptionkey@0"
I20260811 18:58:44.724256 15721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/minicluster-data/ts-0/wal/instance:
uuid: "8d2777c33f3a4f5980f876dba840583c"
format_stamp: "Formatted at 2026-08-11 18:58:44 on dist-test-slave-h6n0"
server_key: "cdd578f600ab0100bb2beb5c06ebc771"
server_key_iv: "9754ea3fcc07fa19ccd2d92ac94c33c5"
server_key_version: "encryptionkey@0"
I20260811 18:58:44.729908 15721 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 18:58:44.733603 15746 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:44.735010 15721 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:58:44.735189 15721 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/minicluster-data/ts-0/wal
uuid: "8d2777c33f3a4f5980f876dba840583c"
format_stamp: "Formatted at 2026-08-11 18:58:44 on dist-test-slave-h6n0"
server_key: "cdd578f600ab0100bb2beb5c06ebc771"
server_key_iv: "9754ea3fcc07fa19ccd2d92ac94c33c5"
server_key_version: "encryptionkey@0"
I20260811 18:58:44.735391 15721 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:58:44.763228 15721 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:58:44.767486 15721 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:58:44.767812 15721 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:58:44.768801 15721 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:58:44.770298 15721 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:58:44.770412 15721 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:44.770561 15721 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:58:44.770633 15721 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260811 18:58:44.807489 15721 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.129:42571
I20260811 18:58:44.807539 15924 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.129:42571 every 8 connection(s)
I20260811 18:58:44.808945 15721 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/minicluster-data/ts-0/data/info.pb
I20260811 18:58:44.809682 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 15721
I20260811 18:58:44.809912 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/minicluster-data/ts-0/wal/instance
I20260811 18:58:44.810806 30146 external_mini_cluster.cc:1428] Setting key e7ff52dc2a812b2a9101c1762cc1ed5b
Aug 11 18:58:44 dist-test-slave-h6n0 krb5kdc[15543](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474724, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.129@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 18:58:44.835569 15929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:44.811895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.129:51133 (local address 127.29.112.190:34387)
0811 18:58:44.812391 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:44.812398 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:44.813837 (+ 1439us) server_negotiation.cc:407] Connection header received
0811 18:58:44.816843 (+ 3006us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:44.816847 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:44.816919 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:44.817020 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:44.819211 (+ 2191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:44.820296 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:44.822970 (+ 2674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:44.823386 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:44.823515 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:44.828379 (+ 4864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:58:44.828389 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:58:44.828397 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:58:44.828443 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:58:44.832123 (+ 3680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:44.832849 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:44.832857 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:44.832864 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:44.833025 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:44.833460 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:44.833468 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:44.833474 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:44.833868 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:58:44.834034 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:44.834802 (+ 768us) server_negotiation.cc:299] Negotiation successful
0811 18:58:44.835083 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":251,"threads_started":1}
I20260811 18:58:44.836163 15930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:44.812514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:34387 (local address 127.29.112.129:51133)
0811 18:58:44.813648 (+ 1134us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:44.813699 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:44.816531 (+ 2832us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:44.817310 (+ 779us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:44.817323 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:44.818118 (+ 795us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:58:44.819000 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:44.819028 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:44.820530 (+ 1502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:44.820538 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:44.822750 (+ 2212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:44.822775 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:44.823642 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:44.823652 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:44.823985 (+ 333us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:44.825269 (+ 1284us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:58:44.825334 (+ 65us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:58:44.828105 (+ 2771us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:58:44.832270 (+ 4165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:58:44.832277 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:58:44.832289 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:58:44.832711 (+ 422us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:58:44.833148 (+ 437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:58:44.833152 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:58:44.833154 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:58:44.833331 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:58:44.834009 (+ 678us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:58:44.834015 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:58:44.834329 (+ 314us) client_negotiation.cc:769] Sending connection context
0811 18:58:44.834644 (+ 315us) client_negotiation.cc:240] Negotiation successful
0811 18:58:44.835053 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":860,"thread_start_us":610,"threads_started":1}
I20260811 18:58:44.837813 15926 heartbeater.cc:347] Connected to a master server at 127.29.112.190:34387
I20260811 18:58:44.838135 15926 heartbeater.cc:464] Registering TS with master...
I20260811 18:58:44.839022 15926 heartbeater.cc:511] Master 127.29.112.190:34387 requested a full tablet report, sending...
I20260811 18:58:44.841193 15619 ts_manager.cc:194] Registered new tserver with Master: 8d2777c33f3a4f5980f876dba840583c (127.29.112.129:42571)
I20260811 18:58:44.842680 15619 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.129@KRBTEST.COM'} at 127.29.112.129:51133
WARNING: no policy specified for kudu/127.29.112.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.130.keytab.
Entry for principal kudu/127.29.112.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.130.keytab.
I20260811 18:58:44.877271 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.130:0
--local_ip_for_outbound_sockets=127.29.112.130
--webserver_interface=127.29.112.130
--webserver_port=0
--tserver_master_addrs=127.29.112.190:34387
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.130.keytab
--principal=kudu/127.29.112.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:38035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:58:45.235143 15934 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:58:45.235396 15934 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:58:45.235565 15934 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:58:45.242774 15934 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:58:45.242871 15934 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:58:45.243076 15934 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.130
I20260811 18:58:45.252280 15934 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:38035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.130.keytab
--principal=kudu/127.29.112.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.112.130
--webserver_port=0
--tserver_master_addrs=127.29.112.190:34387
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.130
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:58:45.253556 15934 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:58:45.254905 15934 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:58:45.268129 15939 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:45.268173 15940 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:45.268196 15942 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:58:45.269186 15934 server_base.cc:1034] running on GCE node
I20260811 18:58:45.269791 15934 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:58:45.270756 15934 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:58:45.271988 15934 hybrid_clock.cc:648] HybridClock initialized: now 1786474725271942 us; error 40 us; skew 500 ppm
I20260811 18:58:45.272313 15934 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:58:45 dist-test-slave-h6n0 krb5kdc[15543](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474725, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:58:45.275688 15934 init.cc:377] Logged in from keytab as kudu/127.29.112.130@KRBTEST.COM (short username kudu)
I20260811 18:58:45.277343 15934 webserver.cc:460] Webserver started at http://127.29.112.130:38491/ using document root <none> and password file <none>
I20260811 18:58:45.278024 15934 fs_manager.cc:359] Metadata directory not provided
I20260811 18:58:45.278147 15934 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:58:45.278448 15934 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:58:45.281584 15934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/minicluster-data/ts-1/data/instance:
uuid: "666da3c6a6a84bb6a5eb9ff2ebcc97b2"
format_stamp: "Formatted at 2026-08-11 18:58:45 on dist-test-slave-h6n0"
server_key: "340a4dc106d14034038d4d10617ceea7"
server_key_iv: "1c269daa9db644579ca1b86598891a40"
server_key_version: "encryptionkey@0"
I20260811 18:58:45.282397 15934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/minicluster-data/ts-1/wal/instance:
uuid: "666da3c6a6a84bb6a5eb9ff2ebcc97b2"
format_stamp: "Formatted at 2026-08-11 18:58:45 on dist-test-slave-h6n0"
server_key: "340a4dc106d14034038d4d10617ceea7"
server_key_iv: "1c269daa9db644579ca1b86598891a40"
server_key_version: "encryptionkey@0"
I20260811 18:58:45.287887 15934 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260811 18:58:45.291402 15952 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:45.292869 15934 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 18:58:45.293025 15934 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/minicluster-data/ts-1/wal
uuid: "666da3c6a6a84bb6a5eb9ff2ebcc97b2"
format_stamp: "Formatted at 2026-08-11 18:58:45 on dist-test-slave-h6n0"
server_key: "340a4dc106d14034038d4d10617ceea7"
server_key_iv: "1c269daa9db644579ca1b86598891a40"
server_key_version: "encryptionkey@0"
I20260811 18:58:45.293203 15934 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:58:45.309715 15934 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:58:45.314132 15934 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:58:45.314445 15934 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:58:45.315801 15934 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:58:45.317338 15934 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:58:45.317437 15934 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:45.317685 15934 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:58:45.317838 15934 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:45.355604 15934 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.130:36543
I20260811 18:58:45.355657 16133 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.130:36543 every 8 connection(s)
I20260811 18:58:45.356853 15934 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/minicluster-data/ts-1/data/info.pb
I20260811 18:58:45.367101 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 15934
I20260811 18:58:45.367304 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/minicluster-data/ts-1/wal/instance
I20260811 18:58:45.367882 30146 external_mini_cluster.cc:1428] Setting key 1e2067eb2cfb6a1e29a7673a4b56c48d
Aug 11 18:58:45 dist-test-slave-h6n0 krb5kdc[15543](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474725, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.130@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 18:58:45.377419 16138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:45.359587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.130:44065 (local address 127.29.112.190:34387)
0811 18:58:45.360121 (+ 534us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:45.360126 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:45.360940 (+ 814us) server_negotiation.cc:407] Connection header received
0811 18:58:45.362758 (+ 1818us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:45.362762 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:45.362829 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:45.362931 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:45.364362 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:45.365046 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:45.366601 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:45.367005 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:45.367098 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:45.371107 (+ 4009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:58:45.371111 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:58:45.371114 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:58:45.371144 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:58:45.374325 (+ 3181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:45.375096 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:45.375107 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:45.375111 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:45.375262 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:45.375719 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:45.375723 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:45.375725 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:45.375968 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:58:45.376150 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:45.376797 (+ 647us) server_negotiation.cc:299] Negotiation successful
0811 18:58:45.377042 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":400,"thread_start_us":277,"threads_started":1}
I20260811 18:58:45.378314 16140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:45.360021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:34387 (local address 127.29.112.130:44065)
0811 18:58:45.360762 (+ 741us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:45.360813 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:45.362447 (+ 1634us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:45.363151 (+ 704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:45.363159 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:45.363655 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:58:45.364213 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:45.364231 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:45.365207 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:45.365211 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:45.366426 (+ 1215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:45.366438 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:45.367259 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:45.367266 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:45.367573 (+ 307us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:45.368326 (+ 753us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:58:45.368356 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:58:45.370887 (+ 2531us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:58:45.374491 (+ 3604us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:58:45.374500 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:58:45.374515 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:58:45.374958 (+ 443us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:58:45.375381 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:58:45.375386 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:58:45.375390 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:58:45.375585 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:58:45.376101 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:58:45.376109 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:58:45.376434 (+ 325us) client_negotiation.cc:769] Sending connection context
0811 18:58:45.376696 (+ 262us) client_negotiation.cc:240] Negotiation successful
0811 18:58:45.377207 (+ 511us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":301,"threads_started":1}
I20260811 18:58:45.379696 16135 heartbeater.cc:347] Connected to a master server at 127.29.112.190:34387
I20260811 18:58:45.379994 16135 heartbeater.cc:464] Registering TS with master...
I20260811 18:58:45.380726 16135 heartbeater.cc:511] Master 127.29.112.190:34387 requested a full tablet report, sending...
I20260811 18:58:45.382128 15619 ts_manager.cc:194] Registered new tserver with Master: 666da3c6a6a84bb6a5eb9ff2ebcc97b2 (127.29.112.130:36543)
I20260811 18:58:45.383106 15619 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.130@KRBTEST.COM'} at 127.29.112.130:44065
WARNING: no policy specified for kudu/127.29.112.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.131.keytab.
Entry for principal kudu/127.29.112.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.131.keytab.
I20260811 18:58:45.432099 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.131:0
--local_ip_for_outbound_sockets=127.29.112.131
--webserver_interface=127.29.112.131
--webserver_port=0
--tserver_master_addrs=127.29.112.190:34387
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.131.keytab
--principal=kudu/127.29.112.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:38035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:58:45.781177 16144 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:58:45.781599 16144 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:58:45.781730 16144 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:58:45.790347 16144 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:58:45.790603 16144 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:58:45.790861 16144 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.131
I20260811 18:58:45.802851 16144 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:38035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.131.keytab
--principal=kudu/127.29.112.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.112.131
--webserver_port=0
--tserver_master_addrs=127.29.112.190:34387
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.131
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:58:45.804136 16144 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:58:45.805385 16144 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:58:45.820360 16157 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:45.820518 16164 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:45.820576 16152 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:58:45.821635 16144 server_base.cc:1034] running on GCE node
I20260811 18:58:45.822242 16144 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:58:45.823215 16144 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:58:45.824455 16144 hybrid_clock.cc:648] HybridClock initialized: now 1786474725824414 us; error 49 us; skew 500 ppm
I20260811 18:58:45.824785 16144 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:58:45 dist-test-slave-h6n0 krb5kdc[15543](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474725, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:58:45.832324 16144 init.cc:377] Logged in from keytab as kudu/127.29.112.131@KRBTEST.COM (short username kudu)
I20260811 18:58:45.833710 16144 webserver.cc:460] Webserver started at http://127.29.112.131:46639/ using document root <none> and password file <none>
I20260811 18:58:45.834403 16144 fs_manager.cc:359] Metadata directory not provided
I20260811 18:58:45.834594 16144 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:58:45.834906 16144 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:58:45.839816 16144 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/minicluster-data/ts-2/data/instance:
uuid: "3e4bc6be157841b8ac537cdf5efb7eb6"
format_stamp: "Formatted at 2026-08-11 18:58:45 on dist-test-slave-h6n0"
server_key: "f665f6242ffea9c6123fd3ad94b7960c"
server_key_iv: "a58b6e6f0bfb5a536e40ea457e9a92f8"
server_key_version: "encryptionkey@0"
I20260811 18:58:45.840754 16144 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/minicluster-data/ts-2/wal/instance:
uuid: "3e4bc6be157841b8ac537cdf5efb7eb6"
format_stamp: "Formatted at 2026-08-11 18:58:45 on dist-test-slave-h6n0"
server_key: "f665f6242ffea9c6123fd3ad94b7960c"
server_key_iv: "a58b6e6f0bfb5a536e40ea457e9a92f8"
server_key_version: "encryptionkey@0"
I20260811 18:58:45.845566 15926 heartbeater.cc:503] Master 127.29.112.190:34387 was elected leader, sending a full tablet report...
I20260811 18:58:45.846081 16144 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.006s
I20260811 18:58:45.849737 16175 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:45.851276 16144 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 18:58:45.851467 16144 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/minicluster-data/ts-2/wal
uuid: "3e4bc6be157841b8ac537cdf5efb7eb6"
format_stamp: "Formatted at 2026-08-11 18:58:45 on dist-test-slave-h6n0"
server_key: "f665f6242ffea9c6123fd3ad94b7960c"
server_key_iv: "a58b6e6f0bfb5a536e40ea457e9a92f8"
server_key_version: "encryptionkey@0"
I20260811 18:58:45.851701 16144 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:58:45.875499 16144 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:58:45.880633 16144 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:58:45.881004 16144 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:58:45.882030 16144 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:58:45.883399 16144 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:58:45.883509 16144 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:45.883663 16144 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:58:45.883738 16144 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:45.924708 16144 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.131:34833
I20260811 18:58:45.924817 16355 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.131:34833 every 8 connection(s)
I20260811 18:58:45.925983 16144 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/minicluster-data/ts-2/data/info.pb
I20260811 18:58:45.932938 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 16144
I20260811 18:58:45.933131 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/minicluster-data/ts-2/wal/instance
I20260811 18:58:45.933694 30146 external_mini_cluster.cc:1428] Setting key dc4fdc0e05d483ec3815f987be9dbc26
Aug 11 18:58:45 dist-test-slave-h6n0 krb5kdc[15543](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474725, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.131@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 18:58:45.947104 16362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:45.928760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.131:58829 (local address 127.29.112.190:34387)
0811 18:58:45.929276 (+ 516us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:45.929280 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:45.930020 (+ 740us) server_negotiation.cc:407] Connection header received
0811 18:58:45.931984 (+ 1964us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:45.931989 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:45.932082 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:45.932202 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:45.934306 (+ 2104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:45.935453 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:45.937460 (+ 2007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:45.937847 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:45.937942 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:45.941672 (+ 3730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:58:45.941677 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:58:45.941680 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:58:45.941710 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:58:45.944395 (+ 2685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:45.945055 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:45.945059 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:45.945061 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:45.945180 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:45.945640 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:45.945643 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:45.945645 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:45.945885 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:58:45.946006 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:45.946690 (+ 684us) server_negotiation.cc:299] Negotiation successful
0811 18:58:45.946846 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":415,"thread_start_us":274,"threads_started":1}
I20260811 18:58:45.948127 16363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:45.929183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:34387 (local address 127.29.112.131:58829)
0811 18:58:45.929862 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:45.929902 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:45.931711 (+ 1809us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:45.932448 (+ 737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:45.932479 (+ 31us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:45.933246 (+ 767us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:58:45.934059 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:45.934088 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:45.935614 (+ 1526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:45.935620 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:45.937321 (+ 1701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:45.937334 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:45.938078 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:45.938084 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:45.938301 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:45.939077 (+ 776us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:58:45.939102 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:58:45.941466 (+ 2364us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:58:45.944535 (+ 3069us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:58:45.944541 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:58:45.944552 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:58:45.944955 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:58:45.945322 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:58:45.945326 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:58:45.945329 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:58:45.945535 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:58:45.946018 (+ 483us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:58:45.946023 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:58:45.946314 (+ 291us) client_negotiation.cc:769] Sending connection context
0811 18:58:45.946627 (+ 313us) client_negotiation.cc:240] Negotiation successful
0811 18:58:45.946991 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":269,"threads_started":1}
I20260811 18:58:45.949607 16356 heartbeater.cc:347] Connected to a master server at 127.29.112.190:34387
I20260811 18:58:45.949945 16356 heartbeater.cc:464] Registering TS with master...
I20260811 18:58:45.950681 16356 heartbeater.cc:511] Master 127.29.112.190:34387 requested a full tablet report, sending...
I20260811 18:58:45.952039 15617 ts_manager.cc:194] Registered new tserver with Master: 3e4bc6be157841b8ac537cdf5efb7eb6 (127.29.112.131:34833)
I20260811 18:58:45.952929 15617 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.131@KRBTEST.COM'} at 127.29.112.131:58829
I20260811 18:58:45.962350 30146 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 18:58:45.980852 16362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:45.967147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54550 (local address 127.29.112.190:34387)
0811 18:58:45.967333 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:45.967337 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:45.967788 (+ 451us) server_negotiation.cc:407] Connection header received
0811 18:58:45.967921 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:45.967924 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:45.968001 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:45.968102 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:45.969591 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:45.970368 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:45.971720 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:45.972169 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:45.972288 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:45.974682 (+ 2394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:58:45.974687 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:58:45.974691 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:58:45.974723 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:58:45.977367 (+ 2644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:45.978079 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:45.978083 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:45.978085 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:45.978213 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:45.978681 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:45.978685 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:45.978687 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:45.979354 (+ 667us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:58:45.979557 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:45.980209 (+ 652us) server_negotiation.cc:299] Negotiation successful
0811 18:58:45.980559 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 18:58:45.985544 15617 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54550:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:58:45.992681 15617 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:58:46.017779 16377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:46.005076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50490 (local address 127.29.112.130:36543)
0811 18:58:46.007663 (+ 2587us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:46.007673 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:46.007706 (+ 33us) server_negotiation.cc:407] Connection header received
0811 18:58:46.007819 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:46.007828 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:46.008148 (+ 320us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:46.009383 (+ 1235us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:58:46.010343 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:46.011732 (+ 1389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:46.014699 (+ 2967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:46.016434 (+ 1735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:46.016604 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:46.016779 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:46.017282 (+ 503us) server_negotiation.cc:299] Negotiation successful
0811 18:58:46.017440 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2436,"thread_start_us":2311,"threads_started":1}
I20260811 18:58:46.017917 16378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:46.005057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.130:36543 (local address 127.0.0.1:50490)
0811 18:58:46.006807 (+ 1750us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:46.006854 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:46.007062 (+ 208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:46.009669 (+ 2607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:46.009678 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:46.009717 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:58:46.010153 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:46.010171 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:46.011893 (+ 1722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:46.011898 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:46.014490 (+ 2592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:46.014507 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:46.016918 (+ 2411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:46.016923 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:46.017044 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:46.017090 (+ 46us) client_negotiation.cc:769] Sending connection context
0811 18:58:46.017231 (+ 141us) client_negotiation.cc:240] Negotiation successful
0811 18:58:46.017427 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1425,"mutex_wait_us":1,"thread_start_us":1266,"threads_started":1}
I20260811 18:58:46.018818 16382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:46.005826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:42571 (local address 127.0.0.1:51468)
0811 18:58:46.008042 (+ 2216us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:46.008074 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:46.008179 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:46.010094 (+ 1915us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:46.010105 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:46.010140 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:58:46.010756 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:46.010775 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:46.012811 (+ 2036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:46.012816 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:46.015535 (+ 2719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:46.015550 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:46.017833 (+ 2283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:46.017846 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:46.018045 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:46.018159 (+ 114us) client_negotiation.cc:769] Sending connection context
0811 18:58:46.018238 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 18:58:46.018433 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2097,"mutex_wait_us":168,"thread_start_us":1945,"threads_started":1}
I20260811 18:58:46.018908 16384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:46.006620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51468 (local address 127.29.112.129:42571)
0811 18:58:46.008772 (+ 2152us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:46.008779 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:46.008804 (+ 25us) server_negotiation.cc:407] Connection header received
0811 18:58:46.008894 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:46.008902 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:46.009119 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:46.010082 (+ 963us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:58:46.010979 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:46.012683 (+ 1704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:46.015690 (+ 3007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:46.017286 (+ 1596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:46.017486 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:46.017643 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:46.018378 (+ 735us) server_negotiation.cc:299] Negotiation successful
0811 18:58:46.018565 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1973,"thread_start_us":1828,"threads_started":1}
I20260811 18:58:46.018908 16387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:46.007795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58780 (local address 127.29.112.131:34833)
0811 18:58:46.009273 (+ 1478us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:46.009285 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:46.009839 (+ 554us) server_negotiation.cc:407] Connection header received
0811 18:58:46.009999 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:46.010004 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:46.010232 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:46.010463 (+ 231us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:58:46.011181 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:46.012870 (+ 1689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:46.015591 (+ 2721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:46.017286 (+ 1695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:46.017487 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:46.017643 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:46.018378 (+ 735us) server_negotiation.cc:299] Negotiation successful
0811 18:58:46.018568 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1162,"thread_start_us":1001,"threads_started":1}
I20260811 18:58:46.019614 16385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:46.007206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.131:34833 (local address 127.0.0.1:58780)
0811 18:58:46.009718 (+ 2512us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:46.009739 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:46.009880 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:46.010462 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:46.010466 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:46.010494 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:58:46.010962 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:46.010975 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:46.013034 (+ 2059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:46.013040 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:46.015413 (+ 2373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:46.015426 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:46.017885 (+ 2459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:46.017892 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:46.018050 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:46.018157 (+ 107us) client_negotiation.cc:769] Sending connection context
0811 18:58:46.018246 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 18:58:46.019207 (+ 961us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2378,"mutex_wait_us":895,"spinlock_wait_cycles":5376,"thread_start_us":2202,"threads_started":1}
I20260811 18:58:46.022274 16026 tablet_service.cc:1467] Processing CreateTablet for tablet b92ae05fdc2c425fa28d5438bfab5161 (DEFAULT_TABLE table=test-table [id=7fbb22bcc2ea4a758b5b955e406da6a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:58:46.022578 15817 tablet_service.cc:1467] Processing CreateTablet for tablet b92ae05fdc2c425fa28d5438bfab5161 (DEFAULT_TABLE table=test-table [id=7fbb22bcc2ea4a758b5b955e406da6a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:58:46.022786 16259 tablet_service.cc:1467] Processing CreateTablet for tablet b92ae05fdc2c425fa28d5438bfab5161 (DEFAULT_TABLE table=test-table [id=7fbb22bcc2ea4a758b5b955e406da6a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:58:46.024065 16026 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b92ae05fdc2c425fa28d5438bfab5161. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:58:46.024065 15817 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b92ae05fdc2c425fa28d5438bfab5161. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:58:46.024312 16259 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b92ae05fdc2c425fa28d5438bfab5161. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:58:46.035512 16389 tablet_bootstrap.cc:492] T b92ae05fdc2c425fa28d5438bfab5161 P 3e4bc6be157841b8ac537cdf5efb7eb6: Bootstrap starting.
I20260811 18:58:46.039404 16391 tablet_bootstrap.cc:492] T b92ae05fdc2c425fa28d5438bfab5161 P 666da3c6a6a84bb6a5eb9ff2ebcc97b2: Bootstrap starting.
I20260811 18:58:46.040213 16389 tablet_bootstrap.cc:654] T b92ae05fdc2c425fa28d5438bfab5161 P 3e4bc6be157841b8ac537cdf5efb7eb6: Neither blocks nor log segments found. Creating new log.
I20260811 18:58:46.041603 16389 log.cc:824] T b92ae05fdc2c425fa28d5438bfab5161 P 3e4bc6be157841b8ac537cdf5efb7eb6: Log is configured to *not* fsync() on all Append() calls
I20260811 18:58:46.044063 16392 tablet_bootstrap.cc:492] T b92ae05fdc2c425fa28d5438bfab5161 P 8d2777c33f3a4f5980f876dba840583c: Bootstrap starting.
I20260811 18:58:46.044425 16391 tablet_bootstrap.cc:654] T b92ae05fdc2c425fa28d5438bfab5161 P 666da3c6a6a84bb6a5eb9ff2ebcc97b2: Neither blocks nor log segments found. Creating new log.
I20260811 18:58:46.045967 16391 log.cc:824] T b92ae05fdc2c425fa28d5438bfab5161 P 666da3c6a6a84bb6a5eb9ff2ebcc97b2: Log is configured to *not* fsync() on all Append() calls
I20260811 18:58:46.046200 16389 tablet_bootstrap.cc:492] T b92ae05fdc2c425fa28d5438bfab5161 P 3e4bc6be157841b8ac537cdf5efb7eb6: No bootstrap required, opened a new log
I20260811 18:58:46.046487 16389 ts_tablet_manager.cc:1397] T b92ae05fdc2c425fa28d5438bfab5161 P 3e4bc6be157841b8ac537cdf5efb7eb6: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.007s
I20260811 18:58:46.047958 16392 tablet_bootstrap.cc:654] T b92ae05fdc2c425fa28d5438bfab5161 P 8d2777c33f3a4f5980f876dba840583c: Neither blocks nor log segments found. Creating new log.
I20260811 18:58:46.049253 16392 log.cc:824] T b92ae05fdc2c425fa28d5438bfab5161 P 8d2777c33f3a4f5980f876dba840583c: Log is configured to *not* fsync() on all Append() calls
I20260811 18:58:46.049654 16391 tablet_bootstrap.cc:492] T b92ae05fdc2c425fa28d5438bfab5161 P 666da3c6a6a84bb6a5eb9ff2ebcc97b2: No bootstrap required, opened a new log
I20260811 18:58:46.049976 16391 ts_tablet_manager.cc:1397] T b92ae05fdc2c425fa28d5438bfab5161 P 666da3c6a6a84bb6a5eb9ff2ebcc97b2: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.004s
I20260811 18:58:46.054185 16389 raft_consensus.cc:348] T b92ae05fdc2c425fa28d5438bfab5161 P 3e4bc6be157841b8ac537cdf5efb7eb6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666da3c6a6a84bb6a5eb9ff2ebcc97b2" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 36543 } } peers { permanent_uuid: "3e4bc6be157841b8ac537cdf5efb7eb6" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 34833 } } peers { permanent_uuid: "8d2777c33f3a4f5980f876dba840583c" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 42571 } }
I20260811 18:58:46.054600 16389 raft_consensus.cc:374] T b92ae05fdc2c425fa28d5438bfab5161 P 3e4bc6be157841b8ac537cdf5efb7eb6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:58:46.054704 16389 raft_consensus.cc:729] T b92ae05fdc2c425fa28d5438bfab5161 P 3e4bc6be157841b8ac537cdf5efb7eb6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e4bc6be157841b8ac537cdf5efb7eb6, State: Initialized, Role: FOLLOWER
I20260811 18:58:46.055537 16389 consensus_queue.cc:260] T b92ae05fdc2c425fa28d5438bfab5161 P 3e4bc6be157841b8ac537cdf5efb7eb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666da3c6a6a84bb6a5eb9ff2ebcc97b2" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 36543 } } peers { permanent_uuid: "3e4bc6be157841b8ac537cdf5efb7eb6" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 34833 } } peers { permanent_uuid: "8d2777c33f3a4f5980f876dba840583c" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 42571 } }
I20260811 18:58:46.057297 16356 heartbeater.cc:503] Master 127.29.112.190:34387 was elected leader, sending a full tablet report...
I20260811 18:58:46.057940 16392 tablet_bootstrap.cc:492] T b92ae05fdc2c425fa28d5438bfab5161 P 8d2777c33f3a4f5980f876dba840583c: No bootstrap required, opened a new log
I20260811 18:58:46.058233 16392 ts_tablet_manager.cc:1397] T b92ae05fdc2c425fa28d5438bfab5161 P 8d2777c33f3a4f5980f876dba840583c: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.008s
I20260811 18:58:46.059005 16389 ts_tablet_manager.cc:1428] T b92ae05fdc2c425fa28d5438bfab5161 P 3e4bc6be157841b8ac537cdf5efb7eb6: Time spent starting tablet: real 0.012s user 0.006s sys 0.005s
I20260811 18:58:46.061206 16391 raft_consensus.cc:348] T b92ae05fdc2c425fa28d5438bfab5161 P 666da3c6a6a84bb6a5eb9ff2ebcc97b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666da3c6a6a84bb6a5eb9ff2ebcc97b2" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 36543 } } peers { permanent_uuid: "3e4bc6be157841b8ac537cdf5efb7eb6" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 34833 } } peers { permanent_uuid: "8d2777c33f3a4f5980f876dba840583c" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 42571 } }
I20260811 18:58:46.061762 16391 raft_consensus.cc:374] T b92ae05fdc2c425fa28d5438bfab5161 P 666da3c6a6a84bb6a5eb9ff2ebcc97b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:58:46.061889 16391 raft_consensus.cc:729] T b92ae05fdc2c425fa28d5438bfab5161 P 666da3c6a6a84bb6a5eb9ff2ebcc97b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 666da3c6a6a84bb6a5eb9ff2ebcc97b2, State: Initialized, Role: FOLLOWER
I20260811 18:58:46.062752 16391 consensus_queue.cc:260] T b92ae05fdc2c425fa28d5438bfab5161 P 666da3c6a6a84bb6a5eb9ff2ebcc97b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666da3c6a6a84bb6a5eb9ff2ebcc97b2" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 36543 } } peers { permanent_uuid: "3e4bc6be157841b8ac537cdf5efb7eb6" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 34833 } } peers { permanent_uuid: "8d2777c33f3a4f5980f876dba840583c" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 42571 } }
I20260811 18:58:46.064074 16135 heartbeater.cc:503] Master 127.29.112.190:34387 was elected leader, sending a full tablet report...
I20260811 18:58:46.064793 16391 ts_tablet_manager.cc:1428] T b92ae05fdc2c425fa28d5438bfab5161 P 666da3c6a6a84bb6a5eb9ff2ebcc97b2: Time spent starting tablet: real 0.015s user 0.010s sys 0.004s
I20260811 18:58:46.070055 16392 raft_consensus.cc:348] T b92ae05fdc2c425fa28d5438bfab5161 P 8d2777c33f3a4f5980f876dba840583c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666da3c6a6a84bb6a5eb9ff2ebcc97b2" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 36543 } } peers { permanent_uuid: "3e4bc6be157841b8ac537cdf5efb7eb6" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 34833 } } peers { permanent_uuid: "8d2777c33f3a4f5980f876dba840583c" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 42571 } }
I20260811 18:58:46.070433 16392 raft_consensus.cc:374] T b92ae05fdc2c425fa28d5438bfab5161 P 8d2777c33f3a4f5980f876dba840583c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:58:46.070588 16392 raft_consensus.cc:729] T b92ae05fdc2c425fa28d5438bfab5161 P 8d2777c33f3a4f5980f876dba840583c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d2777c33f3a4f5980f876dba840583c, State: Initialized, Role: FOLLOWER
I20260811 18:58:46.071141 16392 consensus_queue.cc:260] T b92ae05fdc2c425fa28d5438bfab5161 P 8d2777c33f3a4f5980f876dba840583c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666da3c6a6a84bb6a5eb9ff2ebcc97b2" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 36543 } } peers { permanent_uuid: "3e4bc6be157841b8ac537cdf5efb7eb6" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 34833 } } peers { permanent_uuid: "8d2777c33f3a4f5980f876dba840583c" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 42571 } }
I20260811 18:58:46.072757 16392 ts_tablet_manager.cc:1428] T b92ae05fdc2c425fa28d5438bfab5161 P 8d2777c33f3a4f5980f876dba840583c: Time spent starting tablet: real 0.014s user 0.004s sys 0.011s
I20260811 18:58:46.096657 16401 raft_consensus.cc:482] T b92ae05fdc2c425fa28d5438bfab5161 P 8d2777c33f3a4f5980f876dba840583c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:58:46.097043 16401 raft_consensus.cc:504] T b92ae05fdc2c425fa28d5438bfab5161 P 8d2777c33f3a4f5980f876dba840583c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666da3c6a6a84bb6a5eb9ff2ebcc97b2" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 36543 } } peers { permanent_uuid: "3e4bc6be157841b8ac537cdf5efb7eb6" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 34833 } } peers { permanent_uuid: "8d2777c33f3a4f5980f876dba840583c" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 42571 } }
I20260811 18:58:46.098981 16401 leader_election.cc:288] T b92ae05fdc2c425fa28d5438bfab5161 P 8d2777c33f3a4f5980f876dba840583c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 666da3c6a6a84bb6a5eb9ff2ebcc97b2 (127.29.112.130:36543), 3e4bc6be157841b8ac537cdf5efb7eb6 (127.29.112.131:34833)
I20260811 18:58:46.106348 16377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:46.099316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.129:52921 (local address 127.29.112.130:36543)
0811 18:58:46.099596 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:46.099603 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:46.099975 (+ 372us) server_negotiation.cc:407] Connection header received
0811 18:58:46.100369 (+ 394us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:46.100377 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:46.100475 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:46.100594 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:58:46.101262 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:46.102249 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:46.104296 (+ 2047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:46.105445 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:46.105617 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:46.105693 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:46.105953 (+ 260us) server_negotiation.cc:299] Negotiation successful
0811 18:58:46.106081 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260811 18:58:46.106482 16403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:46.099199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.130:36543 (local address 127.29.112.129:52921)
0811 18:58:46.099873 (+ 674us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:46.099893 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:46.100107 (+ 214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:46.100640 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:46.100648 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:46.100693 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:58:46.101112 (+ 419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:46.101121 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:46.102379 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:46.102383 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:46.104062 (+ 1679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:46.104075 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:46.105644 (+ 1569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:46.105649 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:46.105765 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:46.105789 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 18:58:46.105909 (+ 120us) client_negotiation.cc:240] Negotiation successful
0811 18:58:46.106053 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"mutex_wait_us":1,"thread_start_us":345,"threads_started":1}
I20260811 18:58:46.107187 16387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:46.100130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.129:48995 (local address 127.29.112.131:34833)
0811 18:58:46.100398 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:46.100408 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:46.100939 (+ 531us) server_negotiation.cc:407] Connection header received
0811 18:58:46.101155 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:46.101160 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:46.101262 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:46.101366 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:58:46.101858 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:46.102835 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:46.104777 (+ 1942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:46.106006 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:46.106158 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:46.106243 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:46.106811 (+ 568us) server_negotiation.cc:299] Negotiation successful
0811 18:58:46.106940 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260811 18:58:46.107240 16406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:46.099914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.131:34833 (local address 127.29.112.129:48995)
0811 18:58:46.100840 (+ 926us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:46.100854 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:46.100995 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:46.101404 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:46.101408 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:46.101431 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:58:46.101709 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:46.101716 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:46.102975 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:46.102983 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:46.104603 (+ 1620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:46.104617 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:46.106316 (+ 1699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:46.106321 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:46.106651 (+ 330us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:46.106684 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 18:58:46.106780 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 18:58:46.106903 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":787,"thread_start_us":532,"threads_started":1}
I20260811 18:58:46.107484 16053 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b92ae05fdc2c425fa28d5438bfab5161" candidate_uuid: "8d2777c33f3a4f5980f876dba840583c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666da3c6a6a84bb6a5eb9ff2ebcc97b2" is_pre_election: true
I20260811 18:58:46.108112 16053 raft_consensus.cc:2454] T b92ae05fdc2c425fa28d5438bfab5161 P 666da3c6a6a84bb6a5eb9ff2ebcc97b2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d2777c33f3a4f5980f876dba840583c in term 0.
I20260811 18:58:46.108119 16285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b92ae05fdc2c425fa28d5438bfab5161" candidate_uuid: "8d2777c33f3a4f5980f876dba840583c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e4bc6be157841b8ac537cdf5efb7eb6" is_pre_election: true
I20260811 18:58:46.108646 16285 raft_consensus.cc:2454] T b92ae05fdc2c425fa28d5438bfab5161 P 3e4bc6be157841b8ac537cdf5efb7eb6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d2777c33f3a4f5980f876dba840583c in term 0.
I20260811 18:58:46.108850 15748 leader_election.cc:302] T b92ae05fdc2c425fa28d5438bfab5161 P 8d2777c33f3a4f5980f876dba840583c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 666da3c6a6a84bb6a5eb9ff2ebcc97b2, 8d2777c33f3a4f5980f876dba840583c; no voters:
W20260811 18:58:46.108917 16137 tablet.cc:2369] T b92ae05fdc2c425fa28d5438bfab5161 P 666da3c6a6a84bb6a5eb9ff2ebcc97b2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:58:46.109285 16401 raft_consensus.cc:2781] T b92ae05fdc2c425fa28d5438bfab5161 P 8d2777c33f3a4f5980f876dba840583c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:58:46.109447 16401 raft_consensus.cc:482] T b92ae05fdc2c425fa28d5438bfab5161 P 8d2777c33f3a4f5980f876dba840583c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:58:46.109618 16401 raft_consensus.cc:3037] T b92ae05fdc2c425fa28d5438bfab5161 P 8d2777c33f3a4f5980f876dba840583c [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:58:46.111912 16401 raft_consensus.cc:504] T b92ae05fdc2c425fa28d5438bfab5161 P 8d2777c33f3a4f5980f876dba840583c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666da3c6a6a84bb6a5eb9ff2ebcc97b2" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 36543 } } peers { permanent_uuid: "3e4bc6be157841b8ac537cdf5efb7eb6" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 34833 } } peers { permanent_uuid: "8d2777c33f3a4f5980f876dba840583c" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 42571 } }
I20260811 18:58:46.112510 16401 leader_election.cc:288] T b92ae05fdc2c425fa28d5438bfab5161 P 8d2777c33f3a4f5980f876dba840583c [CANDIDATE]: Term 1 election: Requested vote from peers 666da3c6a6a84bb6a5eb9ff2ebcc97b2 (127.29.112.130:36543), 3e4bc6be157841b8ac537cdf5efb7eb6 (127.29.112.131:34833)
I20260811 18:58:46.113016 16053 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b92ae05fdc2c425fa28d5438bfab5161" candidate_uuid: "8d2777c33f3a4f5980f876dba840583c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666da3c6a6a84bb6a5eb9ff2ebcc97b2"
I20260811 18:58:46.113088 16285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b92ae05fdc2c425fa28d5438bfab5161" candidate_uuid: "8d2777c33f3a4f5980f876dba840583c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e4bc6be157841b8ac537cdf5efb7eb6"
I20260811 18:58:46.113230 16053 raft_consensus.cc:3037] T b92ae05fdc2c425fa28d5438bfab5161 P 666da3c6a6a84bb6a5eb9ff2ebcc97b2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:58:46.113312 16285 raft_consensus.cc:3037] T b92ae05fdc2c425fa28d5438bfab5161 P 3e4bc6be157841b8ac537cdf5efb7eb6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:58:46.115675 16285 raft_consensus.cc:2454] T b92ae05fdc2c425fa28d5438bfab5161 P 3e4bc6be157841b8ac537cdf5efb7eb6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d2777c33f3a4f5980f876dba840583c in term 1.
I20260811 18:58:46.115705 16053 raft_consensus.cc:2454] T b92ae05fdc2c425fa28d5438bfab5161 P 666da3c6a6a84bb6a5eb9ff2ebcc97b2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d2777c33f3a4f5980f876dba840583c in term 1.
I20260811 18:58:46.116096 15748 leader_election.cc:302] T b92ae05fdc2c425fa28d5438bfab5161 P 8d2777c33f3a4f5980f876dba840583c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e4bc6be157841b8ac537cdf5efb7eb6, 8d2777c33f3a4f5980f876dba840583c; no voters:
I20260811 18:58:46.116402 16401 raft_consensus.cc:2781] T b92ae05fdc2c425fa28d5438bfab5161 P 8d2777c33f3a4f5980f876dba840583c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:58:46.116875 16401 raft_consensus.cc:686] T b92ae05fdc2c425fa28d5438bfab5161 P 8d2777c33f3a4f5980f876dba840583c [term 1 LEADER]: Becoming Leader. State: Replica: 8d2777c33f3a4f5980f876dba840583c, State: Running, Role: LEADER
I20260811 18:58:46.117576 16401 consensus_queue.cc:237] T b92ae05fdc2c425fa28d5438bfab5161 P 8d2777c33f3a4f5980f876dba840583c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666da3c6a6a84bb6a5eb9ff2ebcc97b2" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 36543 } } peers { permanent_uuid: "3e4bc6be157841b8ac537cdf5efb7eb6" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 34833 } } peers { permanent_uuid: "8d2777c33f3a4f5980f876dba840583c" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 42571 } }
I20260811 18:58:46.122494 15617 catalog_manager.cc:5495] T b92ae05fdc2c425fa28d5438bfab5161 P 8d2777c33f3a4f5980f876dba840583c reported cstate change: term changed from 0 to 1, leader changed from <none> to 8d2777c33f3a4f5980f876dba840583c (127.29.112.129). New cstate: current_term: 1 leader_uuid: "8d2777c33f3a4f5980f876dba840583c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666da3c6a6a84bb6a5eb9ff2ebcc97b2" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 36543 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e4bc6be157841b8ac537cdf5efb7eb6" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 34833 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8d2777c33f3a4f5980f876dba840583c" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 42571 } health_report { overall_health: HEALTHY } } }
I20260811 18:58:46.140715 16384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:46.135292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51472 (local address 127.29.112.129:42571)
0811 18:58:46.135523 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:46.135530 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:46.135654 (+ 124us) server_negotiation.cc:407] Connection header received
0811 18:58:46.135882 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:46.135886 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:46.135960 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:46.136121 (+ 161us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:58:46.136503 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:46.137213 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:46.138497 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:46.138987 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:46.139127 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:46.139668 (+ 541us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:58:46.140143 (+ 475us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:58:46.140306 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:46.140365 (+ 59us) server_negotiation.cc:299] Negotiation successful
0811 18:58:46.140443 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 18:58:46.148298 16053 raft_consensus.cc:1261] T b92ae05fdc2c425fa28d5438bfab5161 P 666da3c6a6a84bb6a5eb9ff2ebcc97b2 [term 1 FOLLOWER]: Refusing update from remote peer 8d2777c33f3a4f5980f876dba840583c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:58:46.148524 16285 raft_consensus.cc:1261] T b92ae05fdc2c425fa28d5438bfab5161 P 3e4bc6be157841b8ac537cdf5efb7eb6 [term 1 FOLLOWER]: Refusing update from remote peer 8d2777c33f3a4f5980f876dba840583c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:58:46.149293 16407 consensus_queue.cc:1035] T b92ae05fdc2c425fa28d5438bfab5161 P 8d2777c33f3a4f5980f876dba840583c [LEADER]: Connected to new peer: Peer: permanent_uuid: "666da3c6a6a84bb6a5eb9ff2ebcc97b2" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 36543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:58:46.149683 16401 consensus_queue.cc:1035] T b92ae05fdc2c425fa28d5438bfab5161 P 8d2777c33f3a4f5980f876dba840583c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e4bc6be157841b8ac537cdf5efb7eb6" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 34833 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:58:46.186357 15617 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54550:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:58:46.186697 15617 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54550:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:58:46.191367 15617 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0d76b2dbe757456dac0226fe61ed305c: Sending DeleteTablet for 3 replicas of tablet b92ae05fdc2c425fa28d5438bfab5161
I20260811 18:58:46.193101 16026 tablet_service.cc:1514] Processing DeleteTablet for tablet b92ae05fdc2c425fa28d5438bfab5161 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:58:46 UTC) from {username='kudu', principal='kudu/127.29.112.190@KRBTEST.COM'} at 127.0.0.1:50490
I20260811 18:58:46.193313 15817 tablet_service.cc:1514] Processing DeleteTablet for tablet b92ae05fdc2c425fa28d5438bfab5161 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:58:46 UTC) from {username='kudu', principal='kudu/127.29.112.190@KRBTEST.COM'} at 127.0.0.1:51468
I20260811 18:58:46.193948 16259 tablet_service.cc:1514] Processing DeleteTablet for tablet b92ae05fdc2c425fa28d5438bfab5161 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:58:46 UTC) from {username='kudu', principal='kudu/127.29.112.190@KRBTEST.COM'} at 127.0.0.1:58780
I20260811 18:58:46.194370 16428 tablet_replica.cc:330] T b92ae05fdc2c425fa28d5438bfab5161 P 666da3c6a6a84bb6a5eb9ff2ebcc97b2: stopping tablet replica
I20260811 18:58:46.194933 16428 raft_consensus.cc:2229] T b92ae05fdc2c425fa28d5438bfab5161 P 666da3c6a6a84bb6a5eb9ff2ebcc97b2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:58:46.195430 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 15721
I20260811 18:58:46.195693 16428 raft_consensus.cc:2258] T b92ae05fdc2c425fa28d5438bfab5161 P 666da3c6a6a84bb6a5eb9ff2ebcc97b2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:58:46.198040 16428 ts_tablet_manager.cc:1905] T b92ae05fdc2c425fa28d5438bfab5161 P 666da3c6a6a84bb6a5eb9ff2ebcc97b2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:58:46.205355 16428 ts_tablet_manager.cc:1918] T b92ae05fdc2c425fa28d5438bfab5161 P 666da3c6a6a84bb6a5eb9ff2ebcc97b2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:58:46.205608 16428 log.cc:1196] T b92ae05fdc2c425fa28d5438bfab5161 P 666da3c6a6a84bb6a5eb9ff2ebcc97b2: Deleting WAL directory at /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/minicluster-data/ts-1/wal/wals/b92ae05fdc2c425fa28d5438bfab5161
I20260811 18:58:46.206269 16428 ts_tablet_manager.cc:1939] T b92ae05fdc2c425fa28d5438bfab5161 P 666da3c6a6a84bb6a5eb9ff2ebcc97b2: Deleting consensus metadata
I20260811 18:58:46.206413 16430 tablet_replica.cc:330] T b92ae05fdc2c425fa28d5438bfab5161 P 3e4bc6be157841b8ac537cdf5efb7eb6: stopping tablet replica
I20260811 18:58:46.206949 16430 raft_consensus.cc:2229] T b92ae05fdc2c425fa28d5438bfab5161 P 3e4bc6be157841b8ac537cdf5efb7eb6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:58:46.207513 16430 raft_consensus.cc:2258] T b92ae05fdc2c425fa28d5438bfab5161 P 3e4bc6be157841b8ac537cdf5efb7eb6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:58:46.208770 15595 catalog_manager.cc:4841] TS 666da3c6a6a84bb6a5eb9ff2ebcc97b2 (127.29.112.130:36543): tablet b92ae05fdc2c425fa28d5438bfab5161 (table test-table [id=7fbb22bcc2ea4a758b5b955e406da6a6]) successfully deleted
I20260811 18:58:46.210037 16430 ts_tablet_manager.cc:1905] T b92ae05fdc2c425fa28d5438bfab5161 P 3e4bc6be157841b8ac537cdf5efb7eb6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:58:46.211858 16429 tablet_replica.cc:330] T b92ae05fdc2c425fa28d5438bfab5161 P 8d2777c33f3a4f5980f876dba840583c: stopping tablet replica
I20260811 18:58:46.212378 16429 raft_consensus.cc:2229] T b92ae05fdc2c425fa28d5438bfab5161 P 8d2777c33f3a4f5980f876dba840583c [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:58:46.213135 16429 raft_consensus.cc:2258] T b92ae05fdc2c425fa28d5438bfab5161 P 8d2777c33f3a4f5980f876dba840583c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:58:46.215413 16429 ts_tablet_manager.cc:1905] T b92ae05fdc2c425fa28d5438bfab5161 P 8d2777c33f3a4f5980f876dba840583c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:58:46.217080 16430 ts_tablet_manager.cc:1918] T b92ae05fdc2c425fa28d5438bfab5161 P 3e4bc6be157841b8ac537cdf5efb7eb6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:58:46.217252 16430 log.cc:1196] T b92ae05fdc2c425fa28d5438bfab5161 P 3e4bc6be157841b8ac537cdf5efb7eb6: Deleting WAL directory at /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/minicluster-data/ts-2/wal/wals/b92ae05fdc2c425fa28d5438bfab5161
I20260811 18:58:46.217819 16430 ts_tablet_manager.cc:1939] T b92ae05fdc2c425fa28d5438bfab5161 P 3e4bc6be157841b8ac537cdf5efb7eb6: Deleting consensus metadata
I20260811 18:58:46.219801 15595 catalog_manager.cc:4841] TS 3e4bc6be157841b8ac537cdf5efb7eb6 (127.29.112.131:34833): tablet b92ae05fdc2c425fa28d5438bfab5161 (table test-table [id=7fbb22bcc2ea4a758b5b955e406da6a6]) successfully deleted
Aug 11 18:58:46 dist-test-slave-h6n0 krb5kdc[15543](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474723, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.129@KRBTEST.COM
I20260811 18:58:46.223625 16429 ts_tablet_manager.cc:1918] T b92ae05fdc2c425fa28d5438bfab5161 P 8d2777c33f3a4f5980f876dba840583c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:58:46.224042 16429 log.cc:1196] T b92ae05fdc2c425fa28d5438bfab5161 P 8d2777c33f3a4f5980f876dba840583c: Deleting WAL directory at /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474615205622-30146-0/minicluster-data/ts-0/wal/wals/b92ae05fdc2c425fa28d5438bfab5161
I20260811 18:58:46.224907 16429 ts_tablet_manager.cc:1939] T b92ae05fdc2c425fa28d5438bfab5161 P 8d2777c33f3a4f5980f876dba840583c: Deleting consensus metadata
I20260811 18:58:46.226929 15593 catalog_manager.cc:4841] TS 8d2777c33f3a4f5980f876dba840583c (127.29.112.129:42571): tablet b92ae05fdc2c425fa28d5438bfab5161 (table test-table [id=7fbb22bcc2ea4a758b5b955e406da6a6]) successfully deleted
I20260811 18:58:46.230937 16384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:46.197961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51484 (local address 127.29.112.129:42571)
0811 18:58:46.198204 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:46.198214 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:46.211087 (+ 12873us) server_negotiation.cc:407] Connection header received
0811 18:58:46.211310 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:46.211316 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:46.211428 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:46.211561 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:46.213404 (+ 1843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:46.214655 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:46.216475 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:46.217281 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:46.217419 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:46.223130 (+ 5711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:58:46.223160 (+ 30us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:58:46.223181 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:58:46.223241 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:58:46.227952 (+ 4711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:46.228756 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:46.228763 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:46.228766 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:46.228928 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:46.229341 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:46.229348 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:46.229355 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:46.229721 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:58:46.230090 (+ 369us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:46.230324 (+ 234us) server_negotiation.cc:299] Negotiation successful
0811 18:58:46.230598 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 18:58:46.231648 15913 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:58:46.339017 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 15721
I20260811 18:58:46.364701 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 15934
Aug 11 18:58:46 dist-test-slave-h6n0 krb5kdc[15543](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474723, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.130@KRBTEST.COM
I20260811 18:58:46.380479 16377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:46.365549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50498 (local address 127.29.112.130:36543)
0811 18:58:46.365738 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:46.365744 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:46.365765 (+ 21us) server_negotiation.cc:407] Connection header received
0811 18:58:46.365910 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:46.365915 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:46.365993 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:46.366100 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:46.367377 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:46.368180 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:46.369439 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:46.369848 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:46.369987 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:46.374160 (+ 4173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:58:46.374171 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:58:46.374183 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:58:46.374212 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:58:46.377364 (+ 3152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:46.378141 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:46.378149 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:46.378151 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:46.378281 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:46.378754 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:46.378760 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:46.378765 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:46.379148 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:58:46.379472 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:46.379783 (+ 311us) server_negotiation.cc:299] Negotiation successful
0811 18:58:46.380129 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 18:58:46.381219 16126 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:58:46.473047 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 15934
I20260811 18:58:46.501570 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 16144
Aug 11 18:58:46 dist-test-slave-h6n0 krb5kdc[15543](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474723, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.131@KRBTEST.COM
I20260811 18:58:46.517010 16387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:46.502392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58794 (local address 127.29.112.131:34833)
0811 18:58:46.502634 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:46.502638 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:46.502659 (+ 21us) server_negotiation.cc:407] Connection header received
0811 18:58:46.502799 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:46.502803 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:46.502867 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:46.502954 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:46.504286 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:46.505046 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:46.506335 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:46.506878 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:46.507137 (+ 259us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:46.511734 (+ 4597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:58:46.511747 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:58:46.511763 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:58:46.511809 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:58:46.514721 (+ 2912us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:46.515308 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:46.515314 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:46.515317 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:46.515441 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:46.515774 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:46.515777 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:46.515779 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:46.516107 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:58:46.516327 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:46.516614 (+ 287us) server_negotiation.cc:299] Negotiation successful
0811 18:58:46.516772 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 18:58:46.517716 16349 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:58:46.605536 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 16144
I20260811 18:58:46.636333 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 15567
I20260811 18:58:46.637950 15664 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:58:46.751029 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 15567
2026-08-11T18:58:46Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4778 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:58:46 dist-test-slave-h6n0 krb5kdc[16439](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:58:46 dist-test-slave-h6n0 krb5kdc[16439](info): set up 2 sockets
Aug 11 18:58:46 dist-test-slave-h6n0 krb5kdc[16439](info): commencing operation
krb5kdc: starting...
W20260811 18:58:48.135903 30146 mini_kdc.cc:121] Time spent starting KDC: real 1.331s user 0.002s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/krb5kdc/test-user.keytab.
Aug 11 18:58:48 dist-test-slave-h6n0 krb5kdc[16439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474728, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:58:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:58:48Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.29.112.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.29.112.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.29.112.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/krb5kdc/rangeradmin_127.29.112.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.29.112.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/krb5kdc/rangeradmin_127.29.112.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.29.112.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.29.112.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.29.112.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/krb5kdc/rangerlookup_127.29.112.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.29.112.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/krb5kdc/rangerlookup_127.29.112.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.29.112.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.29.112.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.29.112.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/krb5kdc/HTTP_127.29.112.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.29.112.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/krb5kdc/HTTP_127.29.112.148@KRBTEST.COM.keytab.
I20260811 18:58:48.609717 30146 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskFzFlwO/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/minicluster-data/postgres -l logfile start
2026-08-11 18:58:51.328 UTC [16482] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-11 18:58:51.328 UTC [16482] LOG: listening on IPv4 address "127.29.112.148", port 59177
2026-08-11 18:58:51.331 UTC [16482] LOG: listening on Unix socket "/tmp/.s.PGSQL.59177"
2026-08-11 18:58:51.354 UTC [16489] LOG: database system was shut down at 2026-08-11 18:58:49 UTC
2026-08-11 18:58:51.358 UTC [16482] LOG: database system is ready to accept connections
I20260811 18:58:52.596547 30146 mini_postgres.cc:96] Postgres bound to 59177
2026-08-11 18:58:52.621 UTC [16506] FATAL: database "slave" does not exist
127.29.112.148:59177 - accepting connections
I20260811 18:58:52.622761 30146 mini_ranger.cc:161] Starting Ranger...
I20260811 18:58:52.653522 30146 mini_ranger.cc:84] Created miniranger Postgres user
I20260811 18:58:53.139249 30146 mini_ranger.cc:87] Created ranger Postgres database
I20260811 18:58:53.139461 30146 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/minicluster-data/ranger-admin
2026-08-11 18:58:53,486 [I] DB FLAVOR :POSTGRES
2026-08-11 18:58:53,487 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 18:58:53,487 [I] Checking connection..
2026-08-11 18:58:53,487 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFzFlwO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFzFlwO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.112.148:59177/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-11 18:58:53,759 [I] Checking connection passed.
2026-08-11 18:58:53,759 [I] --------- Verifying version history table ---------
2026-08-11 18:58:53,759 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFzFlwO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFzFlwO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.112.148:59177/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 18:58:54,042 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 18:58:54,043 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFzFlwO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFzFlwO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.112.148:59177/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 18:58:54,302 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 18:58:54,303 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-11 18:58:54,303 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFzFlwO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFzFlwO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.112.148:59177/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskFzFlwO/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-11 18:58:54.573 UTC [16615] WARNING: there is no transaction in progress
2026-08-11 18:58:54,584 [I] create_dbversion_catalog.sql file imported successfully
2026-08-11 18:58:54,585 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFzFlwO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFzFlwO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.112.148:59177/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 18:58:54,834 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-11 18:58:54,834 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 18:58:54,835 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFzFlwO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFzFlwO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.112.148:59177/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-11 18:58:55,105 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFzFlwO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFzFlwO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.112.148:59177/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-11 18:58:55,374 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFzFlwO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFzFlwO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.112.148:59177/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-h6n0.c.gcp-upstream.internal','N') ;"
2026-08-11 18:58:55,625 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFzFlwO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFzFlwO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.112.148:59177/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 18:58:55,894 [I] Table x_portal_user does not exist in database ranger
2026-08-11 18:58:55,894 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFzFlwO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFzFlwO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.112.148:59177/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 18:58:56,170 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-11 18:58:56,171 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFzFlwO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFzFlwO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.112.148:59177/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 18:58:56,433 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFzFlwO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFzFlwO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.112.148:59177/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 18:58:56,692 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-11 18:58:56,692 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFzFlwO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFzFlwO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.112.148:59177/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskFzFlwO/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-11 18:58:57.689 UTC [16781] WARNING: there is no transaction in progress
2026-08-11 18:58:57.707 UTC [16781] WARNING: there is no transaction in progress
2026-08-11 18:58:57.724 UTC [16781] WARNING: there is no transaction in progress
2026-08-11 18:58:57.740 UTC [16781] WARNING: there is no transaction in progress
2026-08-11 18:58:57.756 UTC [16781] WARNING: there is no transaction in progress
2026-08-11 18:58:57.777 UTC [16781] WARNING: there is no transaction in progress
2026-08-11 18:58:57.843 UTC [16781] WARNING: there is no transaction in progress
2026-08-11 18:58:57.851 UTC [16781] WARNING: there is no transaction in progress
2026-08-11 18:58:57.867 UTC [16781] WARNING: there is no transaction in progress
2026-08-11 18:58:57.879 UTC [16781] WARNING: there is no transaction in progress
2026-08-11 18:58:57.897 UTC [16781] WARNING: there is no transaction in progress
2026-08-11 18:58:57.916 UTC [16781] WARNING: there is no transaction in progress
2026-08-11 18:58:57.927 UTC [16781] WARNING: there is no transaction in progress
2026-08-11 18:58:57.935 UTC [16781] WARNING: there is no transaction in progress
2026-08-11 18:58:57.942 UTC [16781] WARNING: there is no transaction in progress
2026-08-11 18:58:57.950 UTC [16781] WARNING: there is no transaction in progress
2026-08-11 18:58:58,754 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-11 18:58:58,755 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFzFlwO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFzFlwO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.112.148:59177/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-11 18:58:59,014 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-11 18:58:59,015 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFzFlwO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFzFlwO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.112.148:59177/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 18:58:59,298 [I] Table x_portal_user already exists in database 'ranger'
2026-08-11 18:58:59,298 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFzFlwO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFzFlwO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.112.148:59177/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 18:58:59,552 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-11 18:58:59,553 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFzFlwO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFzFlwO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.112.148:59177/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 18:58:59,834 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFzFlwO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFzFlwO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.112.148:59177/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 18:59:00,102 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFzFlwO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFzFlwO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.112.148:59177/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-h6n0.c.gcp-upstream.internal';"
2026-08-11 18:59:00,376 [I] CORE_DB_SCHEMA import status has been updated
2026-08-11 18:59:00,376 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFzFlwO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFzFlwO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.112.148:59177/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-11 18:59:00,651 [I] DB_PATCHES have already been applied
I20260811 18:59:00.657877 30146 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/minicluster-data/ranger-admin:/tmp/dist-test-taskFzFlwO/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskFzFlwO/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskFzFlwO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFzFlwO/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260811 18:59:00.658000 30146 mini_ranger.cc:193] Using host: 127.29.112.148
I20260811 18:59:00.682508 30146 mini_ranger.cc:227] Ranger admin URL: http://127.29.112.148:50283
Aug 11, 2026 6:59:01 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 6:59:01 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 6:59:01 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/minicluster-data/ranger-admin/ews/webapp
Aug 11, 2026 6:59:01 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 11, 2026 6:59:01 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 11, 2026 6:59:01 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/minicluster-data/ranger-admin/ews/webapp].
Aug 11, 2026 6:59:01 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260811 18:59:02.177201 30146 mini_ranger.cc:160] Time spent starting Ranger: real 9.554s user 0.008s sys 0.062s
Aug 11, 2026 6:59:23 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 11, 2026 6:59:23 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 11, 2026 6:59:23 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 11, 2026 6:59:23 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 11, 2026 6:59:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 11, 2026 6:59:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 11, 2026 6:59:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 11, 2026 6:59:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 11, 2026 6:59:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 11, 2026 6:59:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 11, 2026 6:59:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 11, 2026 6:59:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 11, 2026 6:59:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 11, 2026 6:59:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 11, 2026 6:59:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 11, 2026 6:59:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 11, 2026 6:59:23 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 11, 2026 6:59:24 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260811 18:59:25.033010 30146 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.29.112.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.29.112.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.29.112.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/krb5kdc/rangerkms_127.29.112.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.29.112.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/krb5kdc/rangerkms_127.29.112.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.29.112.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.29.112.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.29.112.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/krb5kdc/HTTP_127.29.112.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.29.112.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/krb5kdc/HTTP_127.29.112.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 11 18:59:25 dist-test-slave-h6n0 krb5kdc[16439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474765, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260811 18:59:25.254953 30146 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260811 18:59:25.279037 30146 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260811 18:59:26.145699 30146 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260811 18:59:26.145853 30146 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/minicluster-data/ranger-kms
I20260811 18:59:26.146028 30146 mini_ranger_kms.cc:227] Using postgres at 127.29.112.148:59177
2026-08-11 18:59:26,746 [I] DB FLAVOR :POSTGRES
2026-08-11 18:59:26,747 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 18:59:26,747 [I] Checking connection
2026-08-11 18:59:26,747 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFzFlwO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.112.148:59177/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-11 18:59:27,030 [I] connection success
2026-08-11 18:59:27,030 [I] --------- Verifying Ranger DB tables ---------
2026-08-11 18:59:27,030 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-11 18:59:27,030 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFzFlwO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.112.148:59177/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-11 18:59:27,286 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-11 18:59:27,287 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 18:59:27,287 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-11 18:59:27,287 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFzFlwO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.112.148:59177/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-11 18:59:27,580 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260811 18:59:28.109862 30146 mini_ranger_kms.cc:310] Created kms service
I20260811 18:59:28.313140 30146 mini_ranger_kms.cc:326] Created kudu user
I20260811 18:59:28.362825 30146 mini_ranger_kms.cc:343] Created rangerkms user
Aug 11 18:59:28 dist-test-slave-h6n0 krb5kdc[16439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474765, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.29.112.148@KRBTEST.COM
I20260811 18:59:28.823809 30146 mini_ranger_kms.cc:384] Added ranger policy
I20260811 18:59:28.825269 30146 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/minicluster-data/ranger-kms:/tmp/dist-test-taskFzFlwO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFzFlwO/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskFzFlwO/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260811 18:59:28.825376 30146 mini_ranger_kms.cc:240] Using host: 127.29.112.148
I20260811 18:59:28.859917 30146 mini_ranger_kms.cc:276] Ranger KMS PID: 17048
I20260811 18:59:28.860157 30146 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.29.112.148:36939
Aug 11, 2026 6:59:30 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 6:59:30 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 6:59:30 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 11, 2026 6:59:30 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 11, 2026 6:59:30 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/minicluster-data/ranger-kms/ews/webapp].
Aug 11, 2026 6:59:30 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 11 18:59:31 dist-test-slave-h6n0 krb5kdc[16439](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786474771, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.29.112.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:59:31.880626 30146 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.626s user 0.151s sys 0.489s
I20260811 18:59:31.880981 30146 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260811 18:59:31.881197 30146 mini_ranger_kms.cc:401] 127.29.112.148:36939/kms/v1/keys
Aug 11 18:59:31 dist-test-slave-h6n0 krb5kdc[16439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474771, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.29.112.148@KRBTEST.COM for HTTP/127.29.112.148@KRBTEST.COM
Aug 11 18:59:32 dist-test-slave-h6n0 krb5kdc[16439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474771, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.29.112.148@KRBTEST.COM for HTTP/127.29.112.148@KRBTEST.COM
Aug 11 18:59:34 dist-test-slave-h6n0 krb5kdc[16439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474774, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.29.112.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab.
Entry for principal kudu/127.29.112.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab.
I20260811 18:59:34.755028 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.190:43135
--webserver_interface=127.29.112.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab
--principal=kudu/127.29.112.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:42583
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.190:43135
--ranger_config_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.112.148:36939/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:59:35.218916 17118 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:59:35.219295 17118 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:59:35.219522 17118 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:59:35.229430 17118 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:59:35.229692 17118 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:59:35.229842 17118 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:59:35.230064 17118 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:59:35.230213 17118 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:59:35.245128 17118 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:42583
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/minicluster-data/master-0/wal
--ranger_kms_url=127.29.112.148:36939/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.29.112.190:43135
--ranger_config_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab
--principal=kudu/127.29.112.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.190:43135
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.112.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:59:35.246896 17118 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:59:35.249197 17118 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:59:35.269059 17125 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:35.269265 17127 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:35.269408 17129 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:59:35.271719 17118 server_base.cc:1034] running on GCE node
I20260811 18:59:35.272833 17118 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:59:35.274658 17118 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:59:35.275977 17118 hybrid_clock.cc:648] HybridClock initialized: now 1786474775275810 us; error 193 us; skew 500 ppm
I20260811 18:59:35.276486 17118 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:59:35 dist-test-slave-h6n0 krb5kdc[16439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474775, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:59:35.280522 17118 init.cc:377] Logged in from keytab as kudu/127.29.112.190@KRBTEST.COM (short username kudu)
I20260811 18:59:35.282317 17118 webserver.cc:460] Webserver started at http://127.29.112.190:40749/ using document root <none> and password file <none>
I20260811 18:59:35.283077 17118 fs_manager.cc:359] Metadata directory not provided
I20260811 18:59:35.283219 17118 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:59:35.283551 17118 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 18:59:35 dist-test-slave-h6n0 krb5kdc[16439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474775, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.190@KRBTEST.COM for HTTP/127.29.112.148@KRBTEST.COM
I20260811 18:59:35.350149 17118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/minicluster-data/master-0/data/instance:
uuid: "af459de68f394cc6a27c50c8cad9d047"
format_stamp: "Formatted at 2026-08-11 18:59:35 on dist-test-slave-h6n0"
server_key: "447f0beab48443232493259d6fced179"
server_key_iv: "ffc241aa2543754050b9c1f2c02b7789"
server_key_version: "kuduclusterkey@0"
I20260811 18:59:35.351372 17118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/minicluster-data/master-0/wal/instance:
uuid: "af459de68f394cc6a27c50c8cad9d047"
format_stamp: "Formatted at 2026-08-11 18:59:35 on dist-test-slave-h6n0"
server_key: "447f0beab48443232493259d6fced179"
server_key_iv: "ffc241aa2543754050b9c1f2c02b7789"
server_key_version: "kuduclusterkey@0"
I20260811 18:59:35.358767 17118 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260811 18:59:35.379297 17138 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:35.381098 17118 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 18:59:35.381294 17118 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/minicluster-data/master-0/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/minicluster-data/master-0/wal
uuid: "af459de68f394cc6a27c50c8cad9d047"
format_stamp: "Formatted at 2026-08-11 18:59:35 on dist-test-slave-h6n0"
server_key: "447f0beab48443232493259d6fced179"
server_key_iv: "ffc241aa2543754050b9c1f2c02b7789"
server_key_version: "kuduclusterkey@0"
I20260811 18:59:35.381549 17118 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:59:35.421324 17118 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:59:35.425653 17118 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:59:35.425994 17118 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:59:35.446081 17118 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.190:43135
I20260811 18:59:35.446170 17221 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.190:43135 every 8 connection(s)
I20260811 18:59:35.447536 17118 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
I20260811 18:59:35.451438 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 17118
I20260811 18:59:35.451782 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/minicluster-data/master-0/wal/instance
I20260811 18:59:35.452793 30146 external_mini_cluster.cc:1428] Setting key 6e5521c09eae69090eb90fb745e4fb53
I20260811 18:59:35.452899 17222 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:59:35.468434 17222 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af459de68f394cc6a27c50c8cad9d047: Bootstrap starting.
I20260811 18:59:35.472555 17222 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P af459de68f394cc6a27c50c8cad9d047: Neither blocks nor log segments found. Creating new log.
Aug 11 18:59:35 dist-test-slave-h6n0 krb5kdc[16439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474774, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 18:59:35.473776 17222 log.cc:824] T 00000000000000000000000000000000 P af459de68f394cc6a27c50c8cad9d047: Log is configured to *not* fsync() on all Append() calls
I20260811 18:59:35.477559 17222 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af459de68f394cc6a27c50c8cad9d047: No bootstrap required, opened a new log
I20260811 18:59:35.482929 17225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:35.455302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40364 (local address 127.29.112.190:43135)
0811 18:59:35.461572 (+ 6270us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:35.461585 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:35.461628 (+ 43us) server_negotiation.cc:407] Connection header received
0811 18:59:35.463499 (+ 1871us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:35.463532 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:35.463936 (+ 404us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:35.464258 (+ 322us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:35.465997 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:35.467377 (+ 1380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:35.469193 (+ 1816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:35.469831 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:35.469998 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:35.474902 (+ 4904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:35.474918 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:35.474936 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:59:35.475000 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:35.478671 (+ 3671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:35.479553 (+ 882us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:35.479559 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:35.479565 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:35.479722 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:35.480148 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:35.480151 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:35.480153 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:35.480733 (+ 580us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:35.480996 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:35.481451 (+ 455us) server_negotiation.cc:299] Negotiation successful
0811 18:59:35.481792 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6006,"thread_start_us":5723,"threads_started":1}
I20260811 18:59:35.486982 17222 raft_consensus.cc:348] T 00000000000000000000000000000000 P af459de68f394cc6a27c50c8cad9d047 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af459de68f394cc6a27c50c8cad9d047" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 43135 } }
I20260811 18:59:35.487386 17222 raft_consensus.cc:374] T 00000000000000000000000000000000 P af459de68f394cc6a27c50c8cad9d047 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:59:35.487512 17222 raft_consensus.cc:729] T 00000000000000000000000000000000 P af459de68f394cc6a27c50c8cad9d047 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af459de68f394cc6a27c50c8cad9d047, State: Initialized, Role: FOLLOWER
I20260811 18:59:35.488122 17222 consensus_queue.cc:260] T 00000000000000000000000000000000 P af459de68f394cc6a27c50c8cad9d047 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af459de68f394cc6a27c50c8cad9d047" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 43135 } }
I20260811 18:59:35.488336 17222 raft_consensus.cc:388] T 00000000000000000000000000000000 P af459de68f394cc6a27c50c8cad9d047 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:59:35.488441 17222 raft_consensus.cc:482] T 00000000000000000000000000000000 P af459de68f394cc6a27c50c8cad9d047 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:59:35.488559 17222 raft_consensus.cc:3037] T 00000000000000000000000000000000 P af459de68f394cc6a27c50c8cad9d047 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:59:35.490672 17222 raft_consensus.cc:504] T 00000000000000000000000000000000 P af459de68f394cc6a27c50c8cad9d047 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af459de68f394cc6a27c50c8cad9d047" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 43135 } }
I20260811 18:59:35.491112 17222 leader_election.cc:302] T 00000000000000000000000000000000 P af459de68f394cc6a27c50c8cad9d047 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: af459de68f394cc6a27c50c8cad9d047; no voters:
I20260811 18:59:35.491632 17222 leader_election.cc:288] T 00000000000000000000000000000000 P af459de68f394cc6a27c50c8cad9d047 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:59:35.491783 17229 raft_consensus.cc:2781] T 00000000000000000000000000000000 P af459de68f394cc6a27c50c8cad9d047 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:59:35.492556 17229 raft_consensus.cc:686] T 00000000000000000000000000000000 P af459de68f394cc6a27c50c8cad9d047 [term 1 LEADER]: Becoming Leader. State: Replica: af459de68f394cc6a27c50c8cad9d047, State: Running, Role: LEADER
I20260811 18:59:35.493077 17229 consensus_queue.cc:237] T 00000000000000000000000000000000 P af459de68f394cc6a27c50c8cad9d047 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af459de68f394cc6a27c50c8cad9d047" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 43135 } }
I20260811 18:59:35.493705 17222 sys_catalog.cc:564] T 00000000000000000000000000000000 P af459de68f394cc6a27c50c8cad9d047 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:59:35.496491 17230 sys_catalog.cc:455] T 00000000000000000000000000000000 P af459de68f394cc6a27c50c8cad9d047 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "af459de68f394cc6a27c50c8cad9d047" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af459de68f394cc6a27c50c8cad9d047" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 43135 } } }
I20260811 18:59:35.496366 17233 sys_catalog.cc:455] T 00000000000000000000000000000000 P af459de68f394cc6a27c50c8cad9d047 [sys.catalog]: SysCatalogTable state changed. Reason: New leader af459de68f394cc6a27c50c8cad9d047. Latest consensus state: current_term: 1 leader_uuid: "af459de68f394cc6a27c50c8cad9d047" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af459de68f394cc6a27c50c8cad9d047" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 43135 } } }
I20260811 18:59:35.496843 17230 sys_catalog.cc:458] T 00000000000000000000000000000000 P af459de68f394cc6a27c50c8cad9d047 [sys.catalog]: This master's current role is: LEADER
I20260811 18:59:35.496869 17233 sys_catalog.cc:458] T 00000000000000000000000000000000 P af459de68f394cc6a27c50c8cad9d047 [sys.catalog]: This master's current role is: LEADER
I20260811 18:59:35.498378 17244 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:59:35.502980 17244 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:59:35.503917 17222 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 18:59:35.519302 17244 catalog_manager.cc:1347] Generated new cluster ID: 400812afb316400f8210bdf0c5fb3662
I20260811 18:59:35.519558 17244 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:59:35.552495 17244 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:59:35.553936 17244 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:59:35.563936 17244 catalog_manager.cc:5868] T 00000000000000000000000000000000 P af459de68f394cc6a27c50c8cad9d047: Generated new TSK 0
I20260811 18:59:35.565256 17244 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.112.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.129.keytab.
Entry for principal kudu/127.29.112.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.129.keytab.
I20260811 18:59:36.463299 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.112.148:36939/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.129:0
--local_ip_for_outbound_sockets=127.29.112.129
--webserver_interface=127.29.112.129
--webserver_port=0
--tserver_master_addrs=127.29.112.190:43135
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.129.keytab
--principal=kudu/127.29.112.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:42583
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 18:59:36 dist-test-slave-h6n0 krb5kdc[16439](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786474776, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260811 18:59:36.981709 17301 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:59:36.982054 17301 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:59:36.982416 17301 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:59:36.993108 17301 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:59:36.993367 17301 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:59:36.993656 17301 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.129
I20260811 18:59:37.011996 17301 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:42583
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.29.112.148:36939/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.129.keytab
--principal=kudu/127.29.112.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.112.129
--webserver_port=0
--tserver_master_addrs=127.29.112.190:43135
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.129
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:59:37.013996 17301 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:59:37.015849 17301 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:59:37.034416 17317 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:37.040519 17318 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:37.040845 17321 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:59:37.040961 17301 server_base.cc:1034] running on GCE node
I20260811 18:59:37.043356 17301 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:59:37.044780 17301 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:59:37.046094 17301 hybrid_clock.cc:648] HybridClock initialized: now 1786474777046022 us; error 60 us; skew 500 ppm
I20260811 18:59:37.046546 17301 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:59:37 dist-test-slave-h6n0 krb5kdc[16439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474777, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:59:37.051227 17301 init.cc:377] Logged in from keytab as kudu/127.29.112.129@KRBTEST.COM (short username kudu)
I20260811 18:59:37.053442 17301 webserver.cc:460] Webserver started at http://127.29.112.129:45247/ using document root <none> and password file <none>
I20260811 18:59:37.054476 17301 fs_manager.cc:359] Metadata directory not provided
I20260811 18:59:37.054718 17301 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:59:37.055169 17301 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 18:59:37 dist-test-slave-h6n0 krb5kdc[16439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474777, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.129@KRBTEST.COM for HTTP/127.29.112.148@KRBTEST.COM
I20260811 18:59:37.084160 17301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/minicluster-data/ts-0/data/instance:
uuid: "f4971333945c44f793cb030325ec1ba2"
format_stamp: "Formatted at 2026-08-11 18:59:37 on dist-test-slave-h6n0"
server_key: "e96b6a31ee3a4af16eac1cfcb953f50e"
server_key_iv: "9fe2de8911f509f97c24c65cf7bbcf6c"
server_key_version: "kuduclusterkey@0"
I20260811 18:59:37.085634 17301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/minicluster-data/ts-0/wal/instance:
uuid: "f4971333945c44f793cb030325ec1ba2"
format_stamp: "Formatted at 2026-08-11 18:59:37 on dist-test-slave-h6n0"
server_key: "e96b6a31ee3a4af16eac1cfcb953f50e"
server_key_iv: "9fe2de8911f509f97c24c65cf7bbcf6c"
server_key_version: "kuduclusterkey@0"
I20260811 18:59:37.099542 17301 fs_manager.cc:691] Time spent creating directory manager: real 0.013s user 0.008s sys 0.004s
I20260811 18:59:37.143527 17334 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:37.145815 17301 fs_manager.cc:725] Time spent opening block manager: real 0.024s user 0.003s sys 0.016s
I20260811 18:59:37.146054 17301 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/minicluster-data/ts-0/wal
uuid: "f4971333945c44f793cb030325ec1ba2"
format_stamp: "Formatted at 2026-08-11 18:59:37 on dist-test-slave-h6n0"
server_key: "e96b6a31ee3a4af16eac1cfcb953f50e"
server_key_iv: "9fe2de8911f509f97c24c65cf7bbcf6c"
server_key_version: "kuduclusterkey@0"
I20260811 18:59:37.146341 17301 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
Aug 11 18:59:37 dist-test-slave-h6n0 krb5kdc[16439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474776, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.190@KRBTEST.COM for HTTP/127.29.112.148@KRBTEST.COM
I20260811 18:59:37.237790 17301 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:59:37.245630 17301 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:59:37.246104 17301 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:59:37.253772 17301 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:59:37.257489 17301 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:59:37.257619 17301 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:37.257809 17301 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:59:37.257973 17301 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:37.309989 17301 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.129:35201
I20260811 18:59:37.310035 17526 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.129:35201 every 8 connection(s)
I20260811 18:59:37.311813 17301 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/minicluster-data/ts-0/data/info.pb
I20260811 18:59:37.319303 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 17301
I20260811 18:59:37.319554 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/minicluster-data/ts-0/wal/instance
I20260811 18:59:37.320537 30146 external_mini_cluster.cc:1428] Setting key c341401bc41060db448636d69379df24
Aug 11 18:59:37 dist-test-slave-h6n0 krb5kdc[16439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474777, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.129@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 18:59:37.350378 17532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:37.314902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.129:56655 (local address 127.29.112.190:43135)
0811 18:59:37.323853 (+ 8951us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:37.323861 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:37.326305 (+ 2444us) server_negotiation.cc:407] Connection header received
0811 18:59:37.329110 (+ 2805us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:37.329119 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:37.329237 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:37.329352 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:37.331710 (+ 2358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:37.332970 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:37.335549 (+ 2579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:37.336277 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:37.336389 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:37.341599 (+ 5210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:37.341608 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:37.341615 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:59:37.341671 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:37.345827 (+ 4156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:37.346910 (+ 1083us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:37.346919 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:37.346925 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:37.347099 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:37.348217 (+ 1118us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:37.348223 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:37.348227 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:37.348594 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:37.348732 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:37.349725 (+ 993us) server_negotiation.cc:299] Negotiation successful
0811 18:59:37.349984 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8775,"thread_start_us":11150,"threads_started":1}
I20260811 18:59:37.352643 17533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:37.315484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:43135 (local address 127.29.112.129:56655)
0811 18:59:37.325740 (+ 10256us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:37.325821 (+ 81us) client_negotiation.cc:174] Beginning negotiation
0811 18:59:37.328649 (+ 2828us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:59:37.329843 (+ 1194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:59:37.329873 (+ 30us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:59:37.330619 (+ 746us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:59:37.331488 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:37.331516 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:37.333178 (+ 1662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:37.333186 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:37.335377 (+ 2191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:37.335395 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:37.336765 (+ 1370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:37.336774 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:37.337136 (+ 362us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:37.338035 (+ 899us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:59:37.338071 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:59:37.341306 (+ 3235us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:59:37.346007 (+ 4701us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:59:37.346017 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:59:37.346036 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:59:37.346770 (+ 734us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:59:37.347247 (+ 477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:59:37.347254 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:59:37.347261 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:59:37.347545 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:59:37.348860 (+ 1315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:59:37.348867 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:59:37.349286 (+ 419us) client_negotiation.cc:769] Sending connection context
0811 18:59:37.350703 (+ 1417us) client_negotiation.cc:240] Negotiation successful
0811 18:59:37.351284 (+ 581us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9894,"thread_start_us":9511,"threads_started":1}
I20260811 18:59:37.354700 17530 heartbeater.cc:347] Connected to a master server at 127.29.112.190:43135
I20260811 18:59:37.355064 17530 heartbeater.cc:464] Registering TS with master...
I20260811 18:59:37.356060 17530 heartbeater.cc:511] Master 127.29.112.190:43135 requested a full tablet report, sending...
I20260811 18:59:37.358804 17163 ts_manager.cc:194] Registered new tserver with Master: f4971333945c44f793cb030325ec1ba2 (127.29.112.129:35201)
WARNING: no policy specified for kudu/127.29.112.130@KRBTEST.COM; defaulting to no policy
I20260811 18:59:37.360594 17163 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.129@KRBTEST.COM'} at 127.29.112.129:56655
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.130.keytab.
Entry for principal kudu/127.29.112.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.130.keytab.
I20260811 18:59:37.399214 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.112.148:36939/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.130:0
--local_ip_for_outbound_sockets=127.29.112.130
--webserver_interface=127.29.112.130
--webserver_port=0
--tserver_master_addrs=127.29.112.190:43135
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.130.keytab
--principal=kudu/127.29.112.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:42583
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 18:59:37 dist-test-slave-h6n0 krb5kdc[16439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474776, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.190@KRBTEST.COM for HTTP/127.29.112.148@KRBTEST.COM
W20260811 18:59:37.885041 17536 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:59:37.885340 17536 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:59:37.885499 17536 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:59:37.893196 17536 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:59:37.893333 17536 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:59:37.893477 17536 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.130
I20260811 18:59:37.904785 17536 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:42583
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.29.112.148:36939/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.130.keytab
--principal=kudu/127.29.112.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.112.130
--webserver_port=0
--tserver_master_addrs=127.29.112.190:43135
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.130
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:59:37.906165 17536 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:59:37.907627 17536 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:59:37.922079 17560 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:37.922214 17557 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:37.922587 17563 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:59:37.923187 17536 server_base.cc:1034] running on GCE node
I20260811 18:59:37.923903 17536 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:59:37.924886 17536 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:59:37.926136 17536 hybrid_clock.cc:648] HybridClock initialized: now 1786474777926104 us; error 58 us; skew 500 ppm
I20260811 18:59:37.926486 17536 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:59:37 dist-test-slave-h6n0 krb5kdc[16439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474777, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:59:37.930008 17536 init.cc:377] Logged in from keytab as kudu/127.29.112.130@KRBTEST.COM (short username kudu)
I20260811 18:59:37.931582 17536 webserver.cc:460] Webserver started at http://127.29.112.130:36117/ using document root <none> and password file <none>
I20260811 18:59:37.932276 17536 fs_manager.cc:359] Metadata directory not provided
I20260811 18:59:37.932417 17536 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:59:37.932746 17536 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 18:59:37 dist-test-slave-h6n0 krb5kdc[16439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474777, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.130@KRBTEST.COM for HTTP/127.29.112.148@KRBTEST.COM
I20260811 18:59:37.947278 17536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/minicluster-data/ts-1/data/instance:
uuid: "d15b889193874dd0898587710604eca3"
format_stamp: "Formatted at 2026-08-11 18:59:37 on dist-test-slave-h6n0"
server_key: "9c19f45fdf6751959d10dfe8588ea771"
server_key_iv: "8950addb33221b5751a04d53a75e7169"
server_key_version: "kuduclusterkey@0"
I20260811 18:59:37.948606 17536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/minicluster-data/ts-1/wal/instance:
uuid: "d15b889193874dd0898587710604eca3"
format_stamp: "Formatted at 2026-08-11 18:59:37 on dist-test-slave-h6n0"
server_key: "9c19f45fdf6751959d10dfe8588ea771"
server_key_iv: "8950addb33221b5751a04d53a75e7169"
server_key_version: "kuduclusterkey@0"
I20260811 18:59:37.954955 17536 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260811 18:59:37.969929 17577 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:37.971803 17536 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260811 18:59:37.972029 17536 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/minicluster-data/ts-1/wal
uuid: "d15b889193874dd0898587710604eca3"
format_stamp: "Formatted at 2026-08-11 18:59:37 on dist-test-slave-h6n0"
server_key: "9c19f45fdf6751959d10dfe8588ea771"
server_key_iv: "8950addb33221b5751a04d53a75e7169"
server_key_version: "kuduclusterkey@0"
I20260811 18:59:37.972299 17536 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:59:37.998703 17536 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:59:38.003283 17536 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:59:38.003685 17536 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:59:38.004853 17536 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:59:38.006588 17536 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:59:38.006692 17536 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260811 18:59:38.006880 17536 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:59:38.006956 17536 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:38.047726 17536 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.130:44809
I20260811 18:59:38.047761 17756 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.130:44809 every 8 connection(s)
I20260811 18:59:38.049113 17536 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/minicluster-data/ts-1/data/info.pb
I20260811 18:59:38.056160 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 17536
I20260811 18:59:38.056381 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/minicluster-data/ts-1/wal/instance
I20260811 18:59:38.056891 30146 external_mini_cluster.cc:1428] Setting key b633de75f54d7bbfb73af5c272a48d5b
Aug 11 18:59:38 dist-test-slave-h6n0 krb5kdc[16439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474777, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.130@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 18:59:38.074867 17763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:38.052691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.130:54133 (local address 127.29.112.190:43135)
0811 18:59:38.053501 (+ 810us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:38.053548 (+ 47us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:38.054622 (+ 1074us) server_negotiation.cc:407] Connection header received
0811 18:59:38.056639 (+ 2017us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:38.056645 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:38.056756 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:38.056883 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:38.058617 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:38.059714 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:38.061424 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:38.062155 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:38.062302 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:38.066389 (+ 4087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:38.066398 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:38.066406 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:59:38.066455 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:38.070700 (+ 4245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:38.071753 (+ 1053us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:38.071760 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:38.071766 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:38.071932 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:38.072472 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:38.072479 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:38.072485 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:38.072918 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:38.073099 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:38.074052 (+ 953us) server_negotiation.cc:299] Negotiation successful
0811 18:59:38.074241 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":554,"thread_start_us":367,"threads_started":1}
I20260811 18:59:38.075861 17764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:38.053530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:43135 (local address 127.29.112.130:54133)
0811 18:59:38.054356 (+ 826us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:38.054427 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0811 18:59:38.056298 (+ 1871us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:59:38.057139 (+ 841us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:59:38.057147 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:59:38.057602 (+ 455us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:59:38.058342 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:38.058368 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:38.059947 (+ 1579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:38.059952 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:38.061254 (+ 1302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:38.061267 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:38.062411 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:38.062417 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:38.062713 (+ 296us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:38.063481 (+ 768us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:59:38.063504 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:59:38.066156 (+ 2652us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:59:38.070890 (+ 4734us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:59:38.070902 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:59:38.070931 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:59:38.071614 (+ 683us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:59:38.072058 (+ 444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:59:38.072065 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:59:38.072071 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:59:38.072350 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:59:38.073086 (+ 736us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:59:38.073097 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:59:38.073583 (+ 486us) client_negotiation.cc:769] Sending connection context
0811 18:59:38.073925 (+ 342us) client_negotiation.cc:240] Negotiation successful
0811 18:59:38.074515 (+ 590us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":578,"thread_start_us":350,"threads_started":1}
I20260811 18:59:38.077548 17757 heartbeater.cc:347] Connected to a master server at 127.29.112.190:43135
I20260811 18:59:38.077971 17757 heartbeater.cc:464] Registering TS with master...
I20260811 18:59:38.078908 17757 heartbeater.cc:511] Master 127.29.112.190:43135 requested a full tablet report, sending...
I20260811 18:59:38.080585 17163 ts_manager.cc:194] Registered new tserver with Master: d15b889193874dd0898587710604eca3 (127.29.112.130:44809)
I20260811 18:59:38.081494 17163 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.130@KRBTEST.COM'} at 127.29.112.130:54133
WARNING: no policy specified for kudu/127.29.112.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.131.keytab.
Entry for principal kudu/127.29.112.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.131.keytab.
I20260811 18:59:38.134949 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.112.148:36939/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.131:0
--local_ip_for_outbound_sockets=127.29.112.131
--webserver_interface=127.29.112.131
--webserver_port=0
--tserver_master_addrs=127.29.112.190:43135
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.131.keytab
--principal=kudu/127.29.112.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:42583
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 18:59:38.364444 17530 heartbeater.cc:503] Master 127.29.112.190:43135 was elected leader, sending a full tablet report...
W20260811 18:59:38.611382 17767 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:59:38.611692 17767 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:59:38.611861 17767 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:59:38.619374 17767 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:59:38.619498 17767 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:59:38.619697 17767 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.131
I20260811 18:59:38.631793 17767 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:42583
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.29.112.148:36939/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.131.keytab
--principal=kudu/127.29.112.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.112.131
--webserver_port=0
--tserver_master_addrs=127.29.112.190:43135
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.131
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 18:59:38.633653 17767 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:59:38.635335 17767 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:59:38.653328 17775 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:38.653362 17776 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:38.653424 17779 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:59:38.655086 17767 server_base.cc:1034] running on GCE node
I20260811 18:59:38.655856 17767 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:59:38.657073 17767 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:59:38.660545 17767 hybrid_clock.cc:648] HybridClock initialized: now 1786474778660421 us; error 102 us; skew 500 ppm
I20260811 18:59:38.661126 17767 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:59:38 dist-test-slave-h6n0 krb5kdc[16439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474778, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:59:38.665776 17767 init.cc:377] Logged in from keytab as kudu/127.29.112.131@KRBTEST.COM (short username kudu)
I20260811 18:59:38.668275 17767 webserver.cc:460] Webserver started at http://127.29.112.131:34949/ using document root <none> and password file <none>
I20260811 18:59:38.669394 17767 fs_manager.cc:359] Metadata directory not provided
I20260811 18:59:38.669603 17767 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:59:38.670192 17767 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 18:59:38 dist-test-slave-h6n0 krb5kdc[16439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474778, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.131@KRBTEST.COM for HTTP/127.29.112.148@KRBTEST.COM
I20260811 18:59:38.693070 17767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/minicluster-data/ts-2/data/instance:
uuid: "20a65da5d0954e2cb9f824fca70a2fbe"
format_stamp: "Formatted at 2026-08-11 18:59:38 on dist-test-slave-h6n0"
server_key: "b6b18ac9a80698e2659ec00625696644"
server_key_iv: "c0e4a9a37d1a0f85c1a1682ef3604b5a"
server_key_version: "kuduclusterkey@0"
I20260811 18:59:38.694844 17767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/minicluster-data/ts-2/wal/instance:
uuid: "20a65da5d0954e2cb9f824fca70a2fbe"
format_stamp: "Formatted at 2026-08-11 18:59:38 on dist-test-slave-h6n0"
server_key: "b6b18ac9a80698e2659ec00625696644"
server_key_iv: "c0e4a9a37d1a0f85c1a1682ef3604b5a"
server_key_version: "kuduclusterkey@0"
I20260811 18:59:38.701239 17767 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.005s
I20260811 18:59:38.716634 17790 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:38.718292 17767 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 18:59:38.718497 17767 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/minicluster-data/ts-2/wal
uuid: "20a65da5d0954e2cb9f824fca70a2fbe"
format_stamp: "Formatted at 2026-08-11 18:59:38 on dist-test-slave-h6n0"
server_key: "b6b18ac9a80698e2659ec00625696644"
server_key_iv: "c0e4a9a37d1a0f85c1a1682ef3604b5a"
server_key_version: "kuduclusterkey@0"
I20260811 18:59:38.718808 17767 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:59:38.738067 17767 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:59:38.742491 17767 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:59:38.742851 17767 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:59:38.743916 17767 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:59:38.745397 17767 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:59:38.745481 17767 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:38.745615 17767 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:59:38.745688 17767 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:38.787591 17767 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.131:34303
I20260811 18:59:38.787685 17959 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.131:34303 every 8 connection(s)
I20260811 18:59:38.789086 17767 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/minicluster-data/ts-2/data/info.pb
I20260811 18:59:38.799187 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 17767
I20260811 18:59:38.799415 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/minicluster-data/ts-2/wal/instance
I20260811 18:59:38.800263 30146 external_mini_cluster.cc:1428] Setting key 9c9ba0e3822cb2c84fb4ea2c0f434c6e
Aug 11 18:59:38 dist-test-slave-h6n0 krb5kdc[16439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474778, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.131@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 18:59:38.812245 17963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:38.792052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.131:32941 (local address 127.29.112.190:43135)
0811 18:59:38.792641 (+ 589us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:38.792646 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:38.793427 (+ 781us) server_negotiation.cc:407] Connection header received
0811 18:59:38.795495 (+ 2068us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:38.795501 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:38.795591 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:38.795714 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:38.797171 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:38.797954 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:38.799753 (+ 1799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:38.800221 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:38.800359 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:38.804879 (+ 4520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:38.804885 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:38.804888 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:59:38.804920 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:38.807745 (+ 2825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:38.808993 (+ 1248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:38.809000 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:38.809003 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:38.809267 (+ 264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:38.809847 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:38.809853 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:38.809856 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:38.810364 (+ 508us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:38.810522 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:38.811602 (+ 1080us) server_negotiation.cc:299] Negotiation successful
0811 18:59:38.811884 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":477,"thread_start_us":352,"threads_started":1}
I20260811 18:59:38.812896 17965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:38.792549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:43135 (local address 127.29.112.131:32941)
0811 18:59:38.793248 (+ 699us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:38.793295 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 18:59:38.795153 (+ 1858us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:59:38.795962 (+ 809us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:59:38.795970 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:59:38.796418 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:59:38.796996 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:38.797013 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:38.798116 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:38.798122 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:38.799605 (+ 1483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:38.799618 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:38.800493 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:38.800501 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:38.800743 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:38.801748 (+ 1005us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:59:38.801772 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:59:38.804657 (+ 2885us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:59:38.808025 (+ 3368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:59:38.808042 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:59:38.808078 (+ 36us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:59:38.808742 (+ 664us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:59:38.809443 (+ 701us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:59:38.809447 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:59:38.809450 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:59:38.809668 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:59:38.810523 (+ 855us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:59:38.810587 (+ 64us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:59:38.811094 (+ 507us) client_negotiation.cc:769] Sending connection context
0811 18:59:38.811353 (+ 259us) client_negotiation.cc:240] Negotiation successful
0811 18:59:38.811757 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":312,"threads_started":1}
I20260811 18:59:38.814519 17960 heartbeater.cc:347] Connected to a master server at 127.29.112.190:43135
I20260811 18:59:38.815152 17960 heartbeater.cc:464] Registering TS with master...
I20260811 18:59:38.816787 17960 heartbeater.cc:511] Master 127.29.112.190:43135 requested a full tablet report, sending...
I20260811 18:59:38.818888 17163 ts_manager.cc:194] Registered new tserver with Master: 20a65da5d0954e2cb9f824fca70a2fbe (127.29.112.131:34303)
I20260811 18:59:38.819898 17163 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.131@KRBTEST.COM'} at 127.29.112.131:32941
I20260811 18:59:38.832134 30146 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 18:59:38.856101 17963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:38.839162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40378 (local address 127.29.112.190:43135)
0811 18:59:38.839475 (+ 313us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:38.839484 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:38.839873 (+ 389us) server_negotiation.cc:407] Connection header received
0811 18:59:38.840142 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:38.840147 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:38.840239 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:38.840367 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:38.842099 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:38.843009 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:38.844593 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:38.845071 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:38.845224 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:38.848830 (+ 3606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:38.848836 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:38.848839 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:59:38.848877 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:38.852550 (+ 3673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:38.853600 (+ 1050us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:38.853607 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:38.853611 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:38.853751 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:38.854175 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:38.854182 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:38.854185 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:38.854494 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:38.854843 (+ 349us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:38.855488 (+ 645us) server_negotiation.cc:299] Negotiation successful
0811 18:59:38.855696 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111}
I20260811 18:59:38.862293 17163 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40378:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:59:38.870784 17163 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:59:38.915750 17979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:38.887426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:35201 (local address 127.0.0.1:34342)
0811 18:59:38.903408 (+ 15982us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:38.903452 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 18:59:38.903872 (+ 420us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:59:38.906207 (+ 2335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:59:38.906221 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:59:38.906278 (+ 57us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:59:38.907853 (+ 1575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:38.907867 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:38.910023 (+ 2156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:38.910030 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:38.912421 (+ 2391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:38.912443 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:38.914442 (+ 1999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:38.914449 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:38.914729 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:38.914821 (+ 92us) client_negotiation.cc:769] Sending connection context
0811 18:59:38.914973 (+ 152us) client_negotiation.cc:240] Negotiation successful
0811 18:59:38.915236 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15695,"mutex_wait_us":3,"thread_start_us":16612,"threads_started":1}
I20260811 18:59:38.916240 17977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:38.886432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.131:34303 (local address 127.0.0.1:47780)
0811 18:59:38.893741 (+ 7309us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:38.893799 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 18:59:38.894047 (+ 248us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:59:38.908657 (+ 14610us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:59:38.908662 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:59:38.908691 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:59:38.909159 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:38.909169 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:38.911250 (+ 2081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:38.911255 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:38.913439 (+ 2184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:38.913451 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:38.915303 (+ 1852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:38.915310 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:38.915533 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:38.915577 (+ 44us) client_negotiation.cc:769] Sending connection context
0811 18:59:38.915656 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 18:59:38.915809 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6964,"mutex_wait_us":55,"thread_start_us":6768,"threads_started":1}
I20260811 18:59:38.916436 17980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:38.887760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34342 (local address 127.29.112.129:35201)
0811 18:59:38.903250 (+ 15490us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:38.903265 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:38.904608 (+ 1343us) server_negotiation.cc:407] Connection header received
0811 18:59:38.904798 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:38.904811 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:38.905357 (+ 546us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:38.905745 (+ 388us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:59:38.908131 (+ 2386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:38.909864 (+ 1733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:38.912621 (+ 2757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:38.914200 (+ 1579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:38.914442 (+ 242us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:38.914704 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:38.915881 (+ 1177us) server_negotiation.cc:299] Negotiation successful
0811 18:59:38.916071 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15196,"thread_start_us":15508,"threads_started":1}
I20260811 18:59:38.917457 17978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:38.886775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47780 (local address 127.29.112.131:34303)
0811 18:59:38.907939 (+ 21164us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:38.907947 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:38.907983 (+ 36us) server_negotiation.cc:407] Connection header received
0811 18:59:38.908103 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:38.908110 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:38.908346 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:38.908665 (+ 319us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:59:38.909369 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:38.911076 (+ 1707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:38.913615 (+ 2539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:38.915027 (+ 1412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:38.916599 (+ 1572us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:38.916750 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:38.916828 (+ 78us) server_negotiation.cc:299] Negotiation successful
0811 18:59:38.917014 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20967,"spinlock_wait_cycles":23808,"thread_start_us":20954,"threads_started":1}
I20260811 18:59:38.917850 17982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:38.900581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.130:44809 (local address 127.0.0.1:51068)
0811 18:59:38.907297 (+ 6716us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:38.907325 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0811 18:59:38.907470 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:59:38.908694 (+ 1224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:59:38.908700 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:59:38.908730 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:59:38.909336 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:38.909353 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:38.911729 (+ 2376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:38.911734 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:38.914039 (+ 2305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:38.914062 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:38.916913 (+ 2851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:38.916921 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:38.917122 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:38.917163 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 18:59:38.917277 (+ 114us) client_negotiation.cc:240] Negotiation successful
0811 18:59:38.917421 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6504,"thread_start_us":6205,"threads_started":1}
I20260811 18:59:38.917872 17981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:38.894600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51068 (local address 127.29.112.130:44809)
0811 18:59:38.907360 (+ 12760us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:38.907370 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:38.907404 (+ 34us) server_negotiation.cc:407] Connection header received
0811 18:59:38.907689 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:38.907698 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:38.908028 (+ 330us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:38.908279 (+ 251us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:59:38.909540 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:38.911093 (+ 1553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:38.914238 (+ 3145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:38.916370 (+ 2132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:38.916544 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:38.916729 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:38.917347 (+ 618us) server_negotiation.cc:299] Negotiation successful
0811 18:59:38.917535 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12552,"thread_start_us":12636,"threads_started":1}
I20260811 18:59:38.921811 17870 tablet_service.cc:1467] Processing CreateTablet for tablet a2421f60ef1e4a8f8fdd4fa7fb81c723 (DEFAULT_TABLE table=test-table [id=b0549eb86888450fa2fe77629edd393b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:59:38.922017 17422 tablet_service.cc:1467] Processing CreateTablet for tablet a2421f60ef1e4a8f8fdd4fa7fb81c723 (DEFAULT_TABLE table=test-table [id=b0549eb86888450fa2fe77629edd393b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:59:38.923480 17870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2421f60ef1e4a8f8fdd4fa7fb81c723. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:59:38.923032 17653 tablet_service.cc:1467] Processing CreateTablet for tablet a2421f60ef1e4a8f8fdd4fa7fb81c723 (DEFAULT_TABLE table=test-table [id=b0549eb86888450fa2fe77629edd393b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:59:38.924711 17422 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2421f60ef1e4a8f8fdd4fa7fb81c723. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:59:38.924991 17653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2421f60ef1e4a8f8fdd4fa7fb81c723. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:59:38.936839 17983 tablet_bootstrap.cc:492] T a2421f60ef1e4a8f8fdd4fa7fb81c723 P 20a65da5d0954e2cb9f824fca70a2fbe: Bootstrap starting.
I20260811 18:59:38.940976 17983 tablet_bootstrap.cc:654] T a2421f60ef1e4a8f8fdd4fa7fb81c723 P 20a65da5d0954e2cb9f824fca70a2fbe: Neither blocks nor log segments found. Creating new log.
I20260811 18:59:38.942061 17984 tablet_bootstrap.cc:492] T a2421f60ef1e4a8f8fdd4fa7fb81c723 P f4971333945c44f793cb030325ec1ba2: Bootstrap starting.
I20260811 18:59:38.942492 17983 log.cc:824] T a2421f60ef1e4a8f8fdd4fa7fb81c723 P 20a65da5d0954e2cb9f824fca70a2fbe: Log is configured to *not* fsync() on all Append() calls
I20260811 18:59:38.943089 17985 tablet_bootstrap.cc:492] T a2421f60ef1e4a8f8fdd4fa7fb81c723 P d15b889193874dd0898587710604eca3: Bootstrap starting.
I20260811 18:59:38.946671 17983 tablet_bootstrap.cc:492] T a2421f60ef1e4a8f8fdd4fa7fb81c723 P 20a65da5d0954e2cb9f824fca70a2fbe: No bootstrap required, opened a new log
I20260811 18:59:38.946959 17983 ts_tablet_manager.cc:1397] T a2421f60ef1e4a8f8fdd4fa7fb81c723 P 20a65da5d0954e2cb9f824fca70a2fbe: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.007s
I20260811 18:59:38.947887 17985 tablet_bootstrap.cc:654] T a2421f60ef1e4a8f8fdd4fa7fb81c723 P d15b889193874dd0898587710604eca3: Neither blocks nor log segments found. Creating new log.
I20260811 18:59:38.948326 17984 tablet_bootstrap.cc:654] T a2421f60ef1e4a8f8fdd4fa7fb81c723 P f4971333945c44f793cb030325ec1ba2: Neither blocks nor log segments found. Creating new log.
I20260811 18:59:38.949410 17985 log.cc:824] T a2421f60ef1e4a8f8fdd4fa7fb81c723 P d15b889193874dd0898587710604eca3: Log is configured to *not* fsync() on all Append() calls
I20260811 18:59:38.949759 17984 log.cc:824] T a2421f60ef1e4a8f8fdd4fa7fb81c723 P f4971333945c44f793cb030325ec1ba2: Log is configured to *not* fsync() on all Append() calls
I20260811 18:59:38.953768 17985 tablet_bootstrap.cc:492] T a2421f60ef1e4a8f8fdd4fa7fb81c723 P d15b889193874dd0898587710604eca3: No bootstrap required, opened a new log
I20260811 18:59:38.954206 17985 ts_tablet_manager.cc:1397] T a2421f60ef1e4a8f8fdd4fa7fb81c723 P d15b889193874dd0898587710604eca3: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.007s
I20260811 18:59:38.956516 17984 tablet_bootstrap.cc:492] T a2421f60ef1e4a8f8fdd4fa7fb81c723 P f4971333945c44f793cb030325ec1ba2: No bootstrap required, opened a new log
I20260811 18:59:38.956996 17984 ts_tablet_manager.cc:1397] T a2421f60ef1e4a8f8fdd4fa7fb81c723 P f4971333945c44f793cb030325ec1ba2: Time spent bootstrapping tablet: real 0.015s user 0.009s sys 0.000s
I20260811 18:59:38.959273 17983 raft_consensus.cc:348] T a2421f60ef1e4a8f8fdd4fa7fb81c723 P 20a65da5d0954e2cb9f824fca70a2fbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20a65da5d0954e2cb9f824fca70a2fbe" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 34303 } } peers { permanent_uuid: "d15b889193874dd0898587710604eca3" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 44809 } } peers { permanent_uuid: "f4971333945c44f793cb030325ec1ba2" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 35201 } }
I20260811 18:59:38.959940 17983 raft_consensus.cc:374] T a2421f60ef1e4a8f8fdd4fa7fb81c723 P 20a65da5d0954e2cb9f824fca70a2fbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:59:38.960070 17983 raft_consensus.cc:729] T a2421f60ef1e4a8f8fdd4fa7fb81c723 P 20a65da5d0954e2cb9f824fca70a2fbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20a65da5d0954e2cb9f824fca70a2fbe, State: Initialized, Role: FOLLOWER
I20260811 18:59:38.960713 17983 consensus_queue.cc:260] T a2421f60ef1e4a8f8fdd4fa7fb81c723 P 20a65da5d0954e2cb9f824fca70a2fbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20a65da5d0954e2cb9f824fca70a2fbe" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 34303 } } peers { permanent_uuid: "d15b889193874dd0898587710604eca3" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 44809 } } peers { permanent_uuid: "f4971333945c44f793cb030325ec1ba2" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 35201 } }
I20260811 18:59:38.961933 17960 heartbeater.cc:503] Master 127.29.112.190:43135 was elected leader, sending a full tablet report...
I20260811 18:59:38.962458 17983 ts_tablet_manager.cc:1428] T a2421f60ef1e4a8f8fdd4fa7fb81c723 P 20a65da5d0954e2cb9f824fca70a2fbe: Time spent starting tablet: real 0.015s user 0.010s sys 0.004s
I20260811 18:59:38.965358 17985 raft_consensus.cc:348] T a2421f60ef1e4a8f8fdd4fa7fb81c723 P d15b889193874dd0898587710604eca3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20a65da5d0954e2cb9f824fca70a2fbe" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 34303 } } peers { permanent_uuid: "d15b889193874dd0898587710604eca3" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 44809 } } peers { permanent_uuid: "f4971333945c44f793cb030325ec1ba2" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 35201 } }
I20260811 18:59:38.965728 17985 raft_consensus.cc:374] T a2421f60ef1e4a8f8fdd4fa7fb81c723 P d15b889193874dd0898587710604eca3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:59:38.965859 17985 raft_consensus.cc:729] T a2421f60ef1e4a8f8fdd4fa7fb81c723 P d15b889193874dd0898587710604eca3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d15b889193874dd0898587710604eca3, State: Initialized, Role: FOLLOWER
I20260811 18:59:38.966640 17985 consensus_queue.cc:260] T a2421f60ef1e4a8f8fdd4fa7fb81c723 P d15b889193874dd0898587710604eca3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20a65da5d0954e2cb9f824fca70a2fbe" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 34303 } } peers { permanent_uuid: "d15b889193874dd0898587710604eca3" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 44809 } } peers { permanent_uuid: "f4971333945c44f793cb030325ec1ba2" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 35201 } }
I20260811 18:59:38.968191 17757 heartbeater.cc:503] Master 127.29.112.190:43135 was elected leader, sending a full tablet report...
I20260811 18:59:38.969835 17985 ts_tablet_manager.cc:1428] T a2421f60ef1e4a8f8fdd4fa7fb81c723 P d15b889193874dd0898587710604eca3: Time spent starting tablet: real 0.016s user 0.014s sys 0.000s
I20260811 18:59:38.969725 17984 raft_consensus.cc:348] T a2421f60ef1e4a8f8fdd4fa7fb81c723 P f4971333945c44f793cb030325ec1ba2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20a65da5d0954e2cb9f824fca70a2fbe" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 34303 } } peers { permanent_uuid: "d15b889193874dd0898587710604eca3" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 44809 } } peers { permanent_uuid: "f4971333945c44f793cb030325ec1ba2" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 35201 } }
I20260811 18:59:38.970248 17984 raft_consensus.cc:374] T a2421f60ef1e4a8f8fdd4fa7fb81c723 P f4971333945c44f793cb030325ec1ba2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:59:38.970367 17984 raft_consensus.cc:729] T a2421f60ef1e4a8f8fdd4fa7fb81c723 P f4971333945c44f793cb030325ec1ba2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4971333945c44f793cb030325ec1ba2, State: Initialized, Role: FOLLOWER
I20260811 18:59:38.971072 17984 consensus_queue.cc:260] T a2421f60ef1e4a8f8fdd4fa7fb81c723 P f4971333945c44f793cb030325ec1ba2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20a65da5d0954e2cb9f824fca70a2fbe" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 34303 } } peers { permanent_uuid: "d15b889193874dd0898587710604eca3" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 44809 } } peers { permanent_uuid: "f4971333945c44f793cb030325ec1ba2" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 35201 } }
I20260811 18:59:38.973169 17984 ts_tablet_manager.cc:1428] T a2421f60ef1e4a8f8fdd4fa7fb81c723 P f4971333945c44f793cb030325ec1ba2: Time spent starting tablet: real 0.016s user 0.014s sys 0.000s
W20260811 18:59:39.040937 17962 tablet.cc:2369] T a2421f60ef1e4a8f8fdd4fa7fb81c723 P 20a65da5d0954e2cb9f824fca70a2fbe: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 18:59:39.051918 17760 tablet.cc:2369] T a2421f60ef1e4a8f8fdd4fa7fb81c723 P d15b889193874dd0898587710604eca3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:59:39.063052 17990 raft_consensus.cc:482] T a2421f60ef1e4a8f8fdd4fa7fb81c723 P 20a65da5d0954e2cb9f824fca70a2fbe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:59:39.063684 17990 raft_consensus.cc:504] T a2421f60ef1e4a8f8fdd4fa7fb81c723 P 20a65da5d0954e2cb9f824fca70a2fbe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20a65da5d0954e2cb9f824fca70a2fbe" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 34303 } } peers { permanent_uuid: "d15b889193874dd0898587710604eca3" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 44809 } } peers { permanent_uuid: "f4971333945c44f793cb030325ec1ba2" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 35201 } }
I20260811 18:59:39.066345 17990 leader_election.cc:288] T a2421f60ef1e4a8f8fdd4fa7fb81c723 P 20a65da5d0954e2cb9f824fca70a2fbe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d15b889193874dd0898587710604eca3 (127.29.112.130:44809), f4971333945c44f793cb030325ec1ba2 (127.29.112.129:35201)
I20260811 18:59:39.074183 17980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:39.066963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.131:42071 (local address 127.29.112.129:35201)
0811 18:59:39.067186 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:39.067190 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:39.067493 (+ 303us) server_negotiation.cc:407] Connection header received
0811 18:59:39.067754 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:39.067758 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:39.067819 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:39.067926 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:59:39.068390 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:39.069184 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:39.072019 (+ 2835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:39.073184 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:39.073407 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:39.073485 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:39.073769 (+ 284us) server_negotiation.cc:299] Negotiation successful
0811 18:59:39.073903 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260811 18:59:39.074183 17965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:39.067192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:35201 (local address 127.29.112.131:42071)
0811 18:59:39.067386 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:39.067404 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 18:59:39.067574 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:59:39.067944 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:59:39.067948 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:59:39.067979 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:59:39.068206 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:39.068215 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:39.069434 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:39.069444 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:39.071807 (+ 2363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:39.071821 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:39.073397 (+ 1576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:39.073403 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:39.073597 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:39.073622 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 18:59:39.073741 (+ 119us) client_negotiation.cc:240] Negotiation successful
0811 18:59:39.073897 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78,"mutex_wait_us":23}
I20260811 18:59:39.075532 17981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:39.066927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.131:50803 (local address 127.29.112.130:44809)
0811 18:59:39.067190 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:39.067197 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:39.068433 (+ 1236us) server_negotiation.cc:407] Connection header received
0811 18:59:39.068665 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:39.068669 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:39.068892 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:39.069308 (+ 416us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:59:39.070651 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:39.071561 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:39.073507 (+ 1946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:39.074519 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:39.074726 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:39.074795 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:39.075177 (+ 382us) server_negotiation.cc:299] Negotiation successful
0811 18:59:39.075294 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 18:59:39.075644 17995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:39.067399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.130:44809 (local address 127.29.112.131:50803)
0811 18:59:39.068312 (+ 913us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:39.068329 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 18:59:39.068499 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:59:39.069317 (+ 818us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:59:39.069322 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:59:39.069350 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:59:39.070398 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:39.070409 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:39.071856 (+ 1447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:39.071883 (+ 27us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:39.073317 (+ 1434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:39.073331 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:39.074766 (+ 1435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:39.074772 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:39.074979 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:39.075009 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 18:59:39.075119 (+ 110us) client_negotiation.cc:240] Negotiation successful
0811 18:59:39.075253 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":793,"mutex_wait_us":35,"thread_start_us":507,"threads_started":1}
I20260811 18:59:39.075408 17455 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2421f60ef1e4a8f8fdd4fa7fb81c723" candidate_uuid: "20a65da5d0954e2cb9f824fca70a2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4971333945c44f793cb030325ec1ba2" is_pre_election: true
I20260811 18:59:39.076081 17455 raft_consensus.cc:2454] T a2421f60ef1e4a8f8fdd4fa7fb81c723 P f4971333945c44f793cb030325ec1ba2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a65da5d0954e2cb9f824fca70a2fbe in term 0.
I20260811 18:59:39.076949 17792 leader_election.cc:302] T a2421f60ef1e4a8f8fdd4fa7fb81c723 P 20a65da5d0954e2cb9f824fca70a2fbe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a65da5d0954e2cb9f824fca70a2fbe, f4971333945c44f793cb030325ec1ba2; no voters:
I20260811 18:59:39.076843 17685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2421f60ef1e4a8f8fdd4fa7fb81c723" candidate_uuid: "20a65da5d0954e2cb9f824fca70a2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d15b889193874dd0898587710604eca3" is_pre_election: true
W20260811 18:59:39.077369 17531 tablet.cc:2369] T a2421f60ef1e4a8f8fdd4fa7fb81c723 P f4971333945c44f793cb030325ec1ba2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:59:39.077445 17990 raft_consensus.cc:2781] T a2421f60ef1e4a8f8fdd4fa7fb81c723 P 20a65da5d0954e2cb9f824fca70a2fbe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:59:39.077425 17685 raft_consensus.cc:2454] T a2421f60ef1e4a8f8fdd4fa7fb81c723 P d15b889193874dd0898587710604eca3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a65da5d0954e2cb9f824fca70a2fbe in term 0.
I20260811 18:59:39.077590 17990 raft_consensus.cc:482] T a2421f60ef1e4a8f8fdd4fa7fb81c723 P 20a65da5d0954e2cb9f824fca70a2fbe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:59:39.077718 17990 raft_consensus.cc:3037] T a2421f60ef1e4a8f8fdd4fa7fb81c723 P 20a65da5d0954e2cb9f824fca70a2fbe [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:59:39.080477 17990 raft_consensus.cc:504] T a2421f60ef1e4a8f8fdd4fa7fb81c723 P 20a65da5d0954e2cb9f824fca70a2fbe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20a65da5d0954e2cb9f824fca70a2fbe" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 34303 } } peers { permanent_uuid: "d15b889193874dd0898587710604eca3" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 44809 } } peers { permanent_uuid: "f4971333945c44f793cb030325ec1ba2" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 35201 } }
I20260811 18:59:39.081552 17990 leader_election.cc:288] T a2421f60ef1e4a8f8fdd4fa7fb81c723 P 20a65da5d0954e2cb9f824fca70a2fbe [CANDIDATE]: Term 1 election: Requested vote from peers d15b889193874dd0898587710604eca3 (127.29.112.130:44809), f4971333945c44f793cb030325ec1ba2 (127.29.112.129:35201)
I20260811 18:59:39.082214 17685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2421f60ef1e4a8f8fdd4fa7fb81c723" candidate_uuid: "20a65da5d0954e2cb9f824fca70a2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d15b889193874dd0898587710604eca3"
I20260811 18:59:39.082289 17455 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2421f60ef1e4a8f8fdd4fa7fb81c723" candidate_uuid: "20a65da5d0954e2cb9f824fca70a2fbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4971333945c44f793cb030325ec1ba2"
I20260811 18:59:39.082453 17685 raft_consensus.cc:3037] T a2421f60ef1e4a8f8fdd4fa7fb81c723 P d15b889193874dd0898587710604eca3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:59:39.082666 17455 raft_consensus.cc:3037] T a2421f60ef1e4a8f8fdd4fa7fb81c723 P f4971333945c44f793cb030325ec1ba2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:59:39.084798 17685 raft_consensus.cc:2454] T a2421f60ef1e4a8f8fdd4fa7fb81c723 P d15b889193874dd0898587710604eca3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a65da5d0954e2cb9f824fca70a2fbe in term 1.
I20260811 18:59:39.085294 17791 leader_election.cc:302] T a2421f60ef1e4a8f8fdd4fa7fb81c723 P 20a65da5d0954e2cb9f824fca70a2fbe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a65da5d0954e2cb9f824fca70a2fbe, d15b889193874dd0898587710604eca3; no voters:
I20260811 18:59:39.085644 17990 raft_consensus.cc:2781] T a2421f60ef1e4a8f8fdd4fa7fb81c723 P 20a65da5d0954e2cb9f824fca70a2fbe [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:59:39.085589 17455 raft_consensus.cc:2454] T a2421f60ef1e4a8f8fdd4fa7fb81c723 P f4971333945c44f793cb030325ec1ba2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a65da5d0954e2cb9f824fca70a2fbe in term 1.
I20260811 18:59:39.086234 17990 raft_consensus.cc:686] T a2421f60ef1e4a8f8fdd4fa7fb81c723 P 20a65da5d0954e2cb9f824fca70a2fbe [term 1 LEADER]: Becoming Leader. State: Replica: 20a65da5d0954e2cb9f824fca70a2fbe, State: Running, Role: LEADER
I20260811 18:59:39.087105 17990 consensus_queue.cc:237] T a2421f60ef1e4a8f8fdd4fa7fb81c723 P 20a65da5d0954e2cb9f824fca70a2fbe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20a65da5d0954e2cb9f824fca70a2fbe" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 34303 } } peers { permanent_uuid: "d15b889193874dd0898587710604eca3" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 44809 } } peers { permanent_uuid: "f4971333945c44f793cb030325ec1ba2" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 35201 } }
I20260811 18:59:39.092374 17161 catalog_manager.cc:5495] T a2421f60ef1e4a8f8fdd4fa7fb81c723 P 20a65da5d0954e2cb9f824fca70a2fbe reported cstate change: term changed from 0 to 1, leader changed from <none> to 20a65da5d0954e2cb9f824fca70a2fbe (127.29.112.131). New cstate: current_term: 1 leader_uuid: "20a65da5d0954e2cb9f824fca70a2fbe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20a65da5d0954e2cb9f824fca70a2fbe" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 34303 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d15b889193874dd0898587710604eca3" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 44809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f4971333945c44f793cb030325ec1ba2" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 35201 } health_report { overall_health: UNKNOWN } } }
I20260811 18:59:39.116757 17978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:39.110636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51734 (local address 127.29.112.131:34303)
0811 18:59:39.110855 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:39.110864 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:39.110944 (+ 80us) server_negotiation.cc:407] Connection header received
0811 18:59:39.111234 (+ 290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:39.111239 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:39.111321 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:39.111495 (+ 174us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:59:39.111911 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:39.112710 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:39.114359 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:39.114874 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:39.115022 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:39.115589 (+ 567us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:59:39.116119 (+ 530us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:59:39.116296 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:39.116359 (+ 63us) server_negotiation.cc:299] Negotiation successful
0811 18:59:39.116459 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 18:59:39.126153 17685 raft_consensus.cc:1261] T a2421f60ef1e4a8f8fdd4fa7fb81c723 P d15b889193874dd0898587710604eca3 [term 1 FOLLOWER]: Refusing update from remote peer 20a65da5d0954e2cb9f824fca70a2fbe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:59:39.126158 17455 raft_consensus.cc:1261] T a2421f60ef1e4a8f8fdd4fa7fb81c723 P f4971333945c44f793cb030325ec1ba2 [term 1 FOLLOWER]: Refusing update from remote peer 20a65da5d0954e2cb9f824fca70a2fbe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:59:39.127288 17996 consensus_queue.cc:1035] T a2421f60ef1e4a8f8fdd4fa7fb81c723 P 20a65da5d0954e2cb9f824fca70a2fbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "d15b889193874dd0898587710604eca3" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 44809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:59:39.127669 17990 consensus_queue.cc:1035] T a2421f60ef1e4a8f8fdd4fa7fb81c723 P 20a65da5d0954e2cb9f824fca70a2fbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "f4971333945c44f793cb030325ec1ba2" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 35201 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:59:39.141633 18001 mvcc.cc:205] Tried to move back new op lower bound from 7317400695290105856 to 7317400695145160704. Current Snapshot: MvccSnapshot[applied={T|T < 7317400695290105856}]
I20260811 18:59:39.158984 17161 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40378:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:59:39.159390 17161 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40378:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:59:39.163405 17161 catalog_manager.cc:5782] T 00000000000000000000000000000000 P af459de68f394cc6a27c50c8cad9d047: Sending DeleteTablet for 3 replicas of tablet a2421f60ef1e4a8f8fdd4fa7fb81c723
I20260811 18:59:39.164711 17870 tablet_service.cc:1514] Processing DeleteTablet for tablet a2421f60ef1e4a8f8fdd4fa7fb81c723 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:59:39 UTC) from {username='kudu', principal='kudu/127.29.112.190@KRBTEST.COM'} at 127.0.0.1:47780
I20260811 18:59:39.164723 17422 tablet_service.cc:1514] Processing DeleteTablet for tablet a2421f60ef1e4a8f8fdd4fa7fb81c723 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:59:39 UTC) from {username='kudu', principal='kudu/127.29.112.190@KRBTEST.COM'} at 127.0.0.1:34342
I20260811 18:59:39.164902 17653 tablet_service.cc:1514] Processing DeleteTablet for tablet a2421f60ef1e4a8f8fdd4fa7fb81c723 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:59:39 UTC) from {username='kudu', principal='kudu/127.29.112.190@KRBTEST.COM'} at 127.0.0.1:51068
I20260811 18:59:39.167550 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 17301
I20260811 18:59:39.181051 18019 tablet_replica.cc:330] T a2421f60ef1e4a8f8fdd4fa7fb81c723 P 20a65da5d0954e2cb9f824fca70a2fbe: stopping tablet replica
I20260811 18:59:39.181712 18019 raft_consensus.cc:2229] T a2421f60ef1e4a8f8fdd4fa7fb81c723 P 20a65da5d0954e2cb9f824fca70a2fbe [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:59:39.182673 18019 raft_consensus.cc:2258] T a2421f60ef1e4a8f8fdd4fa7fb81c723 P 20a65da5d0954e2cb9f824fca70a2fbe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:59:39.186669 18022 tablet_replica.cc:330] T a2421f60ef1e4a8f8fdd4fa7fb81c723 P d15b889193874dd0898587710604eca3: stopping tablet replica
I20260811 18:59:39.187063 18020 tablet_replica.cc:330] T a2421f60ef1e4a8f8fdd4fa7fb81c723 P f4971333945c44f793cb030325ec1ba2: stopping tablet replica
I20260811 18:59:39.187475 18022 raft_consensus.cc:2229] T a2421f60ef1e4a8f8fdd4fa7fb81c723 P d15b889193874dd0898587710604eca3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:59:39.187858 18020 raft_consensus.cc:2229] T a2421f60ef1e4a8f8fdd4fa7fb81c723 P f4971333945c44f793cb030325ec1ba2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:59:39.188271 18022 raft_consensus.cc:2258] T a2421f60ef1e4a8f8fdd4fa7fb81c723 P d15b889193874dd0898587710604eca3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:59:39.188707 18020 raft_consensus.cc:2258] T a2421f60ef1e4a8f8fdd4fa7fb81c723 P f4971333945c44f793cb030325ec1ba2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:59:39.189333 18019 ts_tablet_manager.cc:1905] T a2421f60ef1e4a8f8fdd4fa7fb81c723 P 20a65da5d0954e2cb9f824fca70a2fbe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:59:39.191457 18022 ts_tablet_manager.cc:1905] T a2421f60ef1e4a8f8fdd4fa7fb81c723 P d15b889193874dd0898587710604eca3: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 11 18:59:39 dist-test-slave-h6n0 krb5kdc[16439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474774, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.129@KRBTEST.COM
I20260811 18:59:39.198596 18019 ts_tablet_manager.cc:1918] T a2421f60ef1e4a8f8fdd4fa7fb81c723 P 20a65da5d0954e2cb9f824fca70a2fbe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:59:39.198813 18019 log.cc:1196] T a2421f60ef1e4a8f8fdd4fa7fb81c723 P 20a65da5d0954e2cb9f824fca70a2fbe: Deleting WAL directory at /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/minicluster-data/ts-2/wal/wals/a2421f60ef1e4a8f8fdd4fa7fb81c723
I20260811 18:59:39.199263 18020 ts_tablet_manager.cc:1905] T a2421f60ef1e4a8f8fdd4fa7fb81c723 P f4971333945c44f793cb030325ec1ba2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:59:39.199445 18019 ts_tablet_manager.cc:1939] T a2421f60ef1e4a8f8fdd4fa7fb81c723 P 20a65da5d0954e2cb9f824fca70a2fbe: Deleting consensus metadata
I20260811 18:59:39.201133 18022 ts_tablet_manager.cc:1918] T a2421f60ef1e4a8f8fdd4fa7fb81c723 P d15b889193874dd0898587710604eca3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:59:39.201519 17139 catalog_manager.cc:4841] TS 20a65da5d0954e2cb9f824fca70a2fbe (127.29.112.131:34303): tablet a2421f60ef1e4a8f8fdd4fa7fb81c723 (table test-table [id=b0549eb86888450fa2fe77629edd393b]) successfully deleted
I20260811 18:59:39.201948 18022 log.cc:1196] T a2421f60ef1e4a8f8fdd4fa7fb81c723 P d15b889193874dd0898587710604eca3: Deleting WAL directory at /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/minicluster-data/ts-1/wal/wals/a2421f60ef1e4a8f8fdd4fa7fb81c723
I20260811 18:59:39.202587 18022 ts_tablet_manager.cc:1939] T a2421f60ef1e4a8f8fdd4fa7fb81c723 P d15b889193874dd0898587710604eca3: Deleting consensus metadata
I20260811 18:59:39.204763 17139 catalog_manager.cc:4841] TS d15b889193874dd0898587710604eca3 (127.29.112.130:44809): tablet a2421f60ef1e4a8f8fdd4fa7fb81c723 (table test-table [id=b0549eb86888450fa2fe77629edd393b]) successfully deleted
I20260811 18:59:39.208521 18020 ts_tablet_manager.cc:1918] T a2421f60ef1e4a8f8fdd4fa7fb81c723 P f4971333945c44f793cb030325ec1ba2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:59:39.208734 18020 log.cc:1196] T a2421f60ef1e4a8f8fdd4fa7fb81c723 P f4971333945c44f793cb030325ec1ba2: Deleting WAL directory at /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474615205622-30146-0/minicluster-data/ts-0/wal/wals/a2421f60ef1e4a8f8fdd4fa7fb81c723
I20260811 18:59:39.209263 18020 ts_tablet_manager.cc:1939] T a2421f60ef1e4a8f8fdd4fa7fb81c723 P f4971333945c44f793cb030325ec1ba2: Deleting consensus metadata
I20260811 18:59:39.211467 17140 catalog_manager.cc:4841] TS f4971333945c44f793cb030325ec1ba2 (127.29.112.129:35201): tablet a2421f60ef1e4a8f8fdd4fa7fb81c723 (table test-table [id=b0549eb86888450fa2fe77629edd393b]) successfully deleted
I20260811 18:59:39.211952 17980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:39.169600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40772 (local address 127.29.112.129:35201)
0811 18:59:39.169848 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:39.169858 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:39.179741 (+ 9883us) server_negotiation.cc:407] Connection header received
0811 18:59:39.180301 (+ 560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:39.180314 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:39.180496 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:39.180815 (+ 319us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:39.183727 (+ 2912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:39.185902 (+ 2175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:39.189916 (+ 4014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:39.190687 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:39.190826 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:39.202826 (+ 12000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:39.202843 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:39.202864 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:59:39.202920 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:39.207734 (+ 4814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:39.208887 (+ 1153us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:39.208899 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:39.208903 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:39.209112 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:39.209719 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:39.209724 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:39.209728 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:39.210268 (+ 540us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:39.210659 (+ 391us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:39.211046 (+ 387us) server_negotiation.cc:299] Negotiation successful
0811 18:59:39.211506 (+ 460us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 18:59:39.212767 17519 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:59:39.339274 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 17301
I20260811 18:59:39.373317 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 17536
Aug 11 18:59:39 dist-test-slave-h6n0 krb5kdc[16439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474774, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.130@KRBTEST.COM
I20260811 18:59:39.394174 17981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:39.374469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34348 (local address 127.29.112.130:44809)
0811 18:59:39.374706 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:39.374711 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:39.374732 (+ 21us) server_negotiation.cc:407] Connection header received
0811 18:59:39.374854 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:39.374858 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:39.374929 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:39.375024 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:39.376896 (+ 1872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:39.378177 (+ 1281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:39.380133 (+ 1956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:39.380574 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:39.380680 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:39.387519 (+ 6839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:39.387536 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:39.387553 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:59:39.387598 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:39.390877 (+ 3279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:39.391665 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:39.391680 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:39.391682 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:39.391824 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:39.392254 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:39.392259 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:39.392271 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:39.392667 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:39.393002 (+ 335us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:39.393351 (+ 349us) server_negotiation.cc:299] Negotiation successful
0811 18:59:39.393600 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111}
I20260811 18:59:39.394960 17750 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:59:39.495826 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 17536
I20260811 18:59:39.525537 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 17767
Aug 11 18:59:39 dist-test-slave-h6n0 krb5kdc[16439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474774, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.131@KRBTEST.COM
I20260811 18:59:39.545430 17978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:39.526323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51736 (local address 127.29.112.131:34303)
0811 18:59:39.526635 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:39.526647 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:39.526670 (+ 23us) server_negotiation.cc:407] Connection header received
0811 18:59:39.526745 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:39.526750 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:39.526869 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:39.527003 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:39.528371 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:39.529666 (+ 1295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:39.530967 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:39.531690 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:39.531843 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:39.537076 (+ 5233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:39.537092 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:39.537112 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:59:39.537168 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:39.541975 (+ 4807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:39.542843 (+ 868us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:39.542855 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:39.542859 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:39.543049 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:39.543472 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:39.543480 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:39.543485 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:39.544039 (+ 554us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:39.544395 (+ 356us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:39.544728 (+ 333us) server_negotiation.cc:299] Negotiation successful
0811 18:59:39.544994 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260811 18:59:39.546181 17954 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:59:39.678622 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 17767
I20260811 18:59:39.702564 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 17118
I20260811 18:59:39.704080 17213 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:59:39.820542 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 17118
I20260811 18:59:39.857889 30146 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260811 18:59:40.205027 30146 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260811 18:59:40.205397 30146 mini_ranger.cc:66] Stopping Ranger...
I20260811 18:59:40.562932 30146 mini_ranger.cc:68] Stopped Ranger
2026-08-11 18:59:40.563 UTC [16482] LOG: received smart shutdown request
2026-08-11 18:59:40.567 UTC [16482] LOG: background worker "logical replication launcher" (PID 16500) exited with exit code 1
2026-08-11 18:59:40.568 UTC [16490] LOG: shutting down
2026-08-11 18:59:40.775 UTC [16482] LOG: database system is shut down
2026-08-11T18:59:40Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (54171 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:59:41 dist-test-slave-h6n0 krb5kdc[18042](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:59:41 dist-test-slave-h6n0 krb5kdc[18042](info): set up 2 sockets
Aug 11 18:59:41 dist-test-slave-h6n0 krb5kdc[18042](info): commencing operation
krb5kdc: starting...
W20260811 18:59:42.413262 30146 mini_kdc.cc:121] Time spent starting KDC: real 1.436s user 0.002s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/krb5kdc/test-user.keytab.
Aug 11 18:59:42 dist-test-slave-h6n0 krb5kdc[18042](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474782, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:59:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:59:42Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.29.112.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.29.112.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.29.112.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/krb5kdc/rangeradmin_127.29.112.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.29.112.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/krb5kdc/rangeradmin_127.29.112.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.29.112.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.29.112.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.29.112.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/krb5kdc/rangerlookup_127.29.112.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.29.112.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/krb5kdc/rangerlookup_127.29.112.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.29.112.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.29.112.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.29.112.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/krb5kdc/HTTP_127.29.112.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.29.112.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/krb5kdc/HTTP_127.29.112.148@KRBTEST.COM.keytab.
I20260811 18:59:42.868045 30146 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskFzFlwO/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/postgres -l logfile start
2026-08-11 18:59:45.634 UTC [18093] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-11 18:59:45.635 UTC [18093] LOG: listening on IPv4 address "127.29.112.148", port 33423
2026-08-11 18:59:45.638 UTC [18093] LOG: listening on Unix socket "/tmp/.s.PGSQL.33423"
2026-08-11 18:59:45.664 UTC [18095] LOG: database system was shut down at 2026-08-11 18:59:44 UTC
2026-08-11 18:59:45.668 UTC [18093] LOG: database system is ready to accept connections
I20260811 18:59:46.955003 30146 mini_postgres.cc:96] Postgres bound to 33423
2026-08-11 18:59:46.979 UTC [18111] FATAL: database "slave" does not exist
127.29.112.148:33423 - accepting connections
I20260811 18:59:46.980443 30146 mini_ranger.cc:161] Starting Ranger...
I20260811 18:59:47.012543 30146 mini_ranger.cc:84] Created miniranger Postgres user
I20260811 18:59:47.520819 30146 mini_ranger.cc:87] Created ranger Postgres database
I20260811 18:59:47.520988 30146 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ranger-admin
2026-08-11 18:59:47,840 [I] DB FLAVOR :POSTGRES
2026-08-11 18:59:47,840 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 18:59:47,840 [I] Checking connection..
2026-08-11 18:59:47,840 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFzFlwO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFzFlwO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.112.148:33423/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-11 18:59:48,122 [I] Checking connection passed.
2026-08-11 18:59:48,122 [I] --------- Verifying version history table ---------
2026-08-11 18:59:48,123 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFzFlwO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFzFlwO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.112.148:33423/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 18:59:48,422 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 18:59:48,423 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFzFlwO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFzFlwO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.112.148:33423/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 18:59:48,729 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 18:59:48,730 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-11 18:59:48,730 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFzFlwO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFzFlwO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.112.148:33423/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskFzFlwO/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-11 18:59:49.016 UTC [18202] WARNING: there is no transaction in progress
2026-08-11 18:59:49,028 [I] create_dbversion_catalog.sql file imported successfully
2026-08-11 18:59:49,028 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFzFlwO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFzFlwO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.112.148:33423/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 18:59:49,314 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-11 18:59:49,315 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 18:59:49,315 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFzFlwO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFzFlwO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.112.148:33423/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-11 18:59:49,610 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFzFlwO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFzFlwO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.112.148:33423/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-11 18:59:49,909 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFzFlwO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFzFlwO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.112.148:33423/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-h6n0.c.gcp-upstream.internal','N') ;"
2026-08-11 18:59:50,199 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFzFlwO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFzFlwO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.112.148:33423/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 18:59:50,480 [I] Table x_portal_user does not exist in database ranger
2026-08-11 18:59:50,481 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFzFlwO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFzFlwO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.112.148:33423/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 18:59:50,767 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-11 18:59:50,768 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFzFlwO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFzFlwO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.112.148:33423/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 18:59:51,045 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFzFlwO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFzFlwO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.112.148:33423/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 18:59:51,341 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-11 18:59:51,341 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFzFlwO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFzFlwO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.112.148:33423/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskFzFlwO/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-11 18:59:52.393 UTC [18377] WARNING: there is no transaction in progress
2026-08-11 18:59:52.412 UTC [18377] WARNING: there is no transaction in progress
2026-08-11 18:59:52.428 UTC [18377] WARNING: there is no transaction in progress
2026-08-11 18:59:52.444 UTC [18377] WARNING: there is no transaction in progress
2026-08-11 18:59:52.459 UTC [18377] WARNING: there is no transaction in progress
2026-08-11 18:59:52.475 UTC [18377] WARNING: there is no transaction in progress
2026-08-11 18:59:52.536 UTC [18377] WARNING: there is no transaction in progress
2026-08-11 18:59:52.542 UTC [18377] WARNING: there is no transaction in progress
2026-08-11 18:59:52.556 UTC [18377] WARNING: there is no transaction in progress
2026-08-11 18:59:52.566 UTC [18377] WARNING: there is no transaction in progress
2026-08-11 18:59:52.576 UTC [18377] WARNING: there is no transaction in progress
2026-08-11 18:59:52.591 UTC [18377] WARNING: there is no transaction in progress
2026-08-11 18:59:52.601 UTC [18377] WARNING: there is no transaction in progress
2026-08-11 18:59:52.608 UTC [18377] WARNING: there is no transaction in progress
2026-08-11 18:59:52.615 UTC [18377] WARNING: there is no transaction in progress
2026-08-11 18:59:52.622 UTC [18377] WARNING: there is no transaction in progress
2026-08-11 18:59:53,347 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-11 18:59:53,348 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFzFlwO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFzFlwO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.112.148:33423/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-11 18:59:53,628 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-11 18:59:53,629 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFzFlwO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFzFlwO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.112.148:33423/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 18:59:53,886 [I] Table x_portal_user already exists in database 'ranger'
2026-08-11 18:59:53,886 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFzFlwO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFzFlwO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.112.148:33423/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 18:59:54,163 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-11 18:59:54,163 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFzFlwO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFzFlwO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.112.148:33423/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 18:59:54,434 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFzFlwO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFzFlwO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.112.148:33423/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 18:59:54,682 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFzFlwO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFzFlwO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.112.148:33423/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-h6n0.c.gcp-upstream.internal';"
2026-08-11 18:59:54,941 [I] CORE_DB_SCHEMA import status has been updated
2026-08-11 18:59:54,941 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFzFlwO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFzFlwO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.112.148:33423/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-11 18:59:55,223 [I] DB_PATCHES have already been applied
I20260811 18:59:55.230163 30146 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ranger-admin:/tmp/dist-test-taskFzFlwO/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskFzFlwO/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskFzFlwO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFzFlwO/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260811 18:59:55.230268 30146 mini_ranger.cc:193] Using host: 127.29.112.148
I20260811 18:59:55.252668 30146 mini_ranger.cc:227] Ranger admin URL: http://127.29.112.148:32959
Aug 11, 2026 6:59:55 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 6:59:55 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 6:59:55 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ranger-admin/ews/webapp
Aug 11, 2026 6:59:55 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 11, 2026 6:59:55 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 11, 2026 6:59:56 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ranger-admin/ews/webapp].
Aug 11, 2026 6:59:56 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260811 18:59:56.770691 30146 mini_ranger.cc:160] Time spent starting Ranger: real 9.790s user 0.000s sys 0.062s
Aug 11, 2026 7:00:22 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 11, 2026 7:00:22 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 11, 2026 7:00:22 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 11, 2026 7:00:22 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 11, 2026 7:00:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 11, 2026 7:00:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 11, 2026 7:00:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 11, 2026 7:00:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 11, 2026 7:00:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 11, 2026 7:00:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 11, 2026 7:00:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 11, 2026 7:00:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 11, 2026 7:00:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 11, 2026 7:00:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 11, 2026 7:00:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 11, 2026 7:00:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 11, 2026 7:00:22 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 11, 2026 7:00:23 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260811 19:00:24.572718 30146 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.29.112.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.29.112.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.29.112.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/krb5kdc/rangerkms_127.29.112.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.29.112.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/krb5kdc/rangerkms_127.29.112.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.29.112.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.29.112.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.29.112.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/krb5kdc/HTTP_127.29.112.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.29.112.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/krb5kdc/HTTP_127.29.112.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 11 19:00:24 dist-test-slave-h6n0 krb5kdc[18042](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474824, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260811 19:00:24.842347 30146 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260811 19:00:24.883268 30146 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260811 19:00:26.124773 30146 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260811 19:00:26.125058 30146 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ranger-kms
I20260811 19:00:26.125156 30146 mini_ranger_kms.cc:227] Using postgres at 127.29.112.148:33423
2026-08-11 19:00:26,634 [I] DB FLAVOR :POSTGRES
2026-08-11 19:00:26,634 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 19:00:26,635 [I] Checking connection
2026-08-11 19:00:26,635 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFzFlwO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.112.148:33423/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-11 19:00:27,022 [I] connection success
2026-08-11 19:00:27,023 [I] --------- Verifying Ranger DB tables ---------
2026-08-11 19:00:27,023 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-11 19:00:27,023 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFzFlwO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.112.148:33423/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-11 19:00:27,366 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-11 19:00:27,366 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 19:00:27,366 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-11 19:00:27,366 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFzFlwO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.112.148:33423/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-11 19:00:27,740 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260811 19:00:28.518338 30146 mini_ranger_kms.cc:310] Created kms service
I20260811 19:00:28.873858 30146 mini_ranger_kms.cc:326] Created kudu user
I20260811 19:00:28.955219 30146 mini_ranger_kms.cc:343] Created rangerkms user
Aug 11 19:00:28 dist-test-slave-h6n0 krb5kdc[18042](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474824, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.29.112.148@KRBTEST.COM
I20260811 19:00:29.526062 30146 mini_ranger_kms.cc:384] Added ranger policy
I20260811 19:00:29.526427 30146 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ranger-kms:/tmp/dist-test-taskFzFlwO/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFzFlwO/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskFzFlwO/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260811 19:00:29.526473 30146 mini_ranger_kms.cc:240] Using host: 127.29.112.148
I20260811 19:00:29.557160 30146 mini_ranger_kms.cc:276] Ranger KMS PID: 18663
I20260811 19:00:29.557364 30146 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.29.112.148:48391
Aug 11, 2026 7:00:30 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 7:00:30 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 7:00:30 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 11, 2026 7:00:30 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 11, 2026 7:00:31 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ranger-kms/ews/webapp].
Aug 11, 2026 7:00:31 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 11 19:00:32 dist-test-slave-h6n0 krb5kdc[18042](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786474832, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.29.112.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:00:32.837774 30146 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 7.995s user 0.113s sys 0.443s
I20260811 19:00:32.838057 30146 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260811 19:00:32.838235 30146 mini_ranger_kms.cc:401] 127.29.112.148:48391/kms/v1/keys
Aug 11 19:00:32 dist-test-slave-h6n0 krb5kdc[18042](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474832, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.29.112.148@KRBTEST.COM for HTTP/127.29.112.148@KRBTEST.COM
Aug 11 19:00:33 dist-test-slave-h6n0 krb5kdc[18042](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474832, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.29.112.148@KRBTEST.COM for HTTP/127.29.112.148@KRBTEST.COM
Aug 11 19:00:35 dist-test-slave-h6n0 krb5kdc[18042](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474835, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.29.112.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab.
Entry for principal kudu/127.29.112.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab.
I20260811 19:00:36.083282 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.190:36797
--webserver_interface=127.29.112.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab
--principal=kudu/127.29.112.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:42911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.190:36797
--ranger_config_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.112.148:48391/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:00:36.512563 18746 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:00:36.512924 18746 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:00:36.513019 18746 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:00:36.523106 18746 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:00:36.523273 18746 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:00:36.523401 18746 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:00:36.523506 18746 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:00:36.523600 18746 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:00:36.534880 18746 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:42911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/master-0/wal
--ranger_kms_url=127.29.112.148:48391/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.29.112.190:36797
--ranger_config_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab
--principal=kudu/127.29.112.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.190:36797
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.112.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:00:36.536384 18746 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:00:36.537806 18746 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:00:36.552652 18753 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:36.552533 18758 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:36.552532 18755 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:36.554126 18746 server_base.cc:1034] running on GCE node
I20260811 19:00:36.555281 18746 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:00:36.557106 18746 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:00:36.558410 18746 hybrid_clock.cc:648] HybridClock initialized: now 1786474836558337 us; error 65 us; skew 500 ppm
I20260811 19:00:36.558907 18746 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:00:36 dist-test-slave-h6n0 krb5kdc[18042](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474836, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:00:36.562831 18746 init.cc:377] Logged in from keytab as kudu/127.29.112.190@KRBTEST.COM (short username kudu)
I20260811 19:00:36.564921 18746 webserver.cc:460] Webserver started at http://127.29.112.190:36799/ using document root <none> and password file <none>
I20260811 19:00:36.565618 18746 fs_manager.cc:359] Metadata directory not provided
I20260811 19:00:36.565729 18746 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:00:36.565995 18746 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 19:00:36 dist-test-slave-h6n0 krb5kdc[18042](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474836, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.190@KRBTEST.COM for HTTP/127.29.112.148@KRBTEST.COM
I20260811 19:00:36.636909 18746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/master-0/data/instance:
uuid: "daba4ff62b2d4177855a7a8bc0ed5bde"
format_stamp: "Formatted at 2026-08-11 19:00:36 on dist-test-slave-h6n0"
server_key: "04d8d55189a02f6212c8a1767fd8bfdd"
server_key_iv: "480e2d1f2fba0f32ebc52ffb1644e2d2"
server_key_version: "kuduclusterkey@0"
I20260811 19:00:36.638072 18746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/master-0/wal/instance:
uuid: "daba4ff62b2d4177855a7a8bc0ed5bde"
format_stamp: "Formatted at 2026-08-11 19:00:36 on dist-test-slave-h6n0"
server_key: "04d8d55189a02f6212c8a1767fd8bfdd"
server_key_iv: "480e2d1f2fba0f32ebc52ffb1644e2d2"
server_key_version: "kuduclusterkey@0"
I20260811 19:00:36.646086 18746 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260811 19:00:36.669430 18768 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:36.671300 18746 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.004s
I20260811 19:00:36.671545 18746 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/master-0/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/master-0/wal
uuid: "daba4ff62b2d4177855a7a8bc0ed5bde"
format_stamp: "Formatted at 2026-08-11 19:00:36 on dist-test-slave-h6n0"
server_key: "04d8d55189a02f6212c8a1767fd8bfdd"
server_key_iv: "480e2d1f2fba0f32ebc52ffb1644e2d2"
server_key_version: "kuduclusterkey@0"
I20260811 19:00:36.671788 18746 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:00:36.698827 18746 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:00:36.703914 18746 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:00:36.704319 18746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:00:36.730258 18746 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.190:36797
I20260811 19:00:36.730310 18848 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.190:36797 every 8 connection(s)
I20260811 19:00:36.731937 18746 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
I20260811 19:00:36.737654 18851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:00:36.739462 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 18746
I20260811 19:00:36.739784 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/master-0/wal/instance
I20260811 19:00:36.740564 30146 external_mini_cluster.cc:1428] Setting key 2ef2ff7ba38a054838e28b5c55f295f7
I20260811 19:00:36.754232 18851 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P daba4ff62b2d4177855a7a8bc0ed5bde: Bootstrap starting.
I20260811 19:00:36.757789 18851 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P daba4ff62b2d4177855a7a8bc0ed5bde: Neither blocks nor log segments found. Creating new log.
Aug 11 19:00:36 dist-test-slave-h6n0 krb5kdc[18042](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474835, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 19:00:36.759150 18851 log.cc:824] T 00000000000000000000000000000000 P daba4ff62b2d4177855a7a8bc0ed5bde: Log is configured to *not* fsync() on all Append() calls
I20260811 19:00:36.762663 18851 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P daba4ff62b2d4177855a7a8bc0ed5bde: No bootstrap required, opened a new log
I20260811 19:00:36.768724 18859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:36.743442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41656 (local address 127.29.112.190:36797)
0811 19:00:36.744396 (+ 954us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:36.744412 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:36.744468 (+ 56us) server_negotiation.cc:407] Connection header received
0811 19:00:36.746520 (+ 2052us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:36.746573 (+ 53us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:36.747140 (+ 567us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:36.747592 (+ 452us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:36.749397 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:36.751602 (+ 2205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:36.753427 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:36.754452 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:36.754675 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:36.760170 (+ 5495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:36.760184 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:36.760211 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:36.760263 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:36.763810 (+ 3547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:36.764839 (+ 1029us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:36.764846 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:36.764852 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:36.765014 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:36.765618 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:36.765631 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:36.765633 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:36.766231 (+ 598us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:36.766625 (+ 394us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:36.767152 (+ 527us) server_negotiation.cc:299] Negotiation successful
0811 19:00:36.767593 (+ 441us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":690,"thread_start_us":394,"threads_started":1}
I20260811 19:00:36.774025 18851 raft_consensus.cc:348] T 00000000000000000000000000000000 P daba4ff62b2d4177855a7a8bc0ed5bde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "daba4ff62b2d4177855a7a8bc0ed5bde" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 36797 } }
I20260811 19:00:36.774603 18851 raft_consensus.cc:374] T 00000000000000000000000000000000 P daba4ff62b2d4177855a7a8bc0ed5bde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:00:36.774879 18851 raft_consensus.cc:729] T 00000000000000000000000000000000 P daba4ff62b2d4177855a7a8bc0ed5bde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: daba4ff62b2d4177855a7a8bc0ed5bde, State: Initialized, Role: FOLLOWER
I20260811 19:00:36.775502 18851 consensus_queue.cc:260] T 00000000000000000000000000000000 P daba4ff62b2d4177855a7a8bc0ed5bde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "daba4ff62b2d4177855a7a8bc0ed5bde" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 36797 } }
I20260811 19:00:36.775804 18851 raft_consensus.cc:388] T 00000000000000000000000000000000 P daba4ff62b2d4177855a7a8bc0ed5bde [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:00:36.775949 18851 raft_consensus.cc:482] T 00000000000000000000000000000000 P daba4ff62b2d4177855a7a8bc0ed5bde [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:00:36.776149 18851 raft_consensus.cc:3037] T 00000000000000000000000000000000 P daba4ff62b2d4177855a7a8bc0ed5bde [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:00:36.778975 18851 raft_consensus.cc:504] T 00000000000000000000000000000000 P daba4ff62b2d4177855a7a8bc0ed5bde [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "daba4ff62b2d4177855a7a8bc0ed5bde" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 36797 } }
I20260811 19:00:36.780318 18851 leader_election.cc:302] T 00000000000000000000000000000000 P daba4ff62b2d4177855a7a8bc0ed5bde [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: daba4ff62b2d4177855a7a8bc0ed5bde; no voters:
I20260811 19:00:36.781278 18851 leader_election.cc:288] T 00000000000000000000000000000000 P daba4ff62b2d4177855a7a8bc0ed5bde [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:00:36.781492 18861 raft_consensus.cc:2781] T 00000000000000000000000000000000 P daba4ff62b2d4177855a7a8bc0ed5bde [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:00:36.782693 18861 raft_consensus.cc:686] T 00000000000000000000000000000000 P daba4ff62b2d4177855a7a8bc0ed5bde [term 1 LEADER]: Becoming Leader. State: Replica: daba4ff62b2d4177855a7a8bc0ed5bde, State: Running, Role: LEADER
I20260811 19:00:36.783500 18861 consensus_queue.cc:237] T 00000000000000000000000000000000 P daba4ff62b2d4177855a7a8bc0ed5bde [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "daba4ff62b2d4177855a7a8bc0ed5bde" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 36797 } }
I20260811 19:00:36.784134 18851 sys_catalog.cc:564] T 00000000000000000000000000000000 P daba4ff62b2d4177855a7a8bc0ed5bde [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:00:36.786741 18863 sys_catalog.cc:455] T 00000000000000000000000000000000 P daba4ff62b2d4177855a7a8bc0ed5bde [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "daba4ff62b2d4177855a7a8bc0ed5bde" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "daba4ff62b2d4177855a7a8bc0ed5bde" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 36797 } } }
I20260811 19:00:36.787282 18863 sys_catalog.cc:458] T 00000000000000000000000000000000 P daba4ff62b2d4177855a7a8bc0ed5bde [sys.catalog]: This master's current role is: LEADER
I20260811 19:00:36.788388 18864 sys_catalog.cc:455] T 00000000000000000000000000000000 P daba4ff62b2d4177855a7a8bc0ed5bde [sys.catalog]: SysCatalogTable state changed. Reason: New leader daba4ff62b2d4177855a7a8bc0ed5bde. Latest consensus state: current_term: 1 leader_uuid: "daba4ff62b2d4177855a7a8bc0ed5bde" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "daba4ff62b2d4177855a7a8bc0ed5bde" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 36797 } } }
I20260811 19:00:36.788897 18864 sys_catalog.cc:458] T 00000000000000000000000000000000 P daba4ff62b2d4177855a7a8bc0ed5bde [sys.catalog]: This master's current role is: LEADER
I20260811 19:00:36.791010 18868 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:00:36.800180 18868 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:00:36.801101 18851 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 19:00:36.825767 18868 catalog_manager.cc:1347] Generated new cluster ID: 763fcf9fa3bb4ee7b3dbce487ec850bf
I20260811 19:00:36.825939 18868 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:00:36.874963 18868 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:00:36.877475 18868 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:00:36.895659 18868 catalog_manager.cc:5868] T 00000000000000000000000000000000 P daba4ff62b2d4177855a7a8bc0ed5bde: Generated new TSK 0
I20260811 19:00:36.896814 18868 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.112.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.129.keytab.
Entry for principal kudu/127.29.112.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.129.keytab.
I20260811 19:00:37.780974 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.112.148:48391/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.129:0
--local_ip_for_outbound_sockets=127.29.112.129
--webserver_interface=127.29.112.129
--webserver_port=0
--tserver_master_addrs=127.29.112.190:36797
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.129.keytab
--principal=kudu/127.29.112.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:42911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 19:00:38 dist-test-slave-h6n0 krb5kdc[18042](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786474838, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260811 19:00:38.335052 18920 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:00:38.335707 18920 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:00:38.336395 18920 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:00:38.348228 18920 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:00:38.348452 18920 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:00:38.348667 18920 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.129
I20260811 19:00:38.366847 18920 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:42911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.29.112.148:48391/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.129.keytab
--principal=kudu/127.29.112.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.112.129
--webserver_port=0
--tserver_master_addrs=127.29.112.190:36797
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.129
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:00:38.368599 18920 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:00:38.370108 18920 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:00:38.395210 18939 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:38.396483 18937 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:38.405220 18920 server_base.cc:1034] running on GCE node
W20260811 19:00:38.405761 18942 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:38.407387 18920 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:00:38.408828 18920 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:00:38.410153 18920 hybrid_clock.cc:648] HybridClock initialized: now 1786474838410066 us; error 65 us; skew 500 ppm
I20260811 19:00:38.410707 18920 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:00:38 dist-test-slave-h6n0 krb5kdc[18042](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474838, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:00:38.417546 18920 init.cc:377] Logged in from keytab as kudu/127.29.112.129@KRBTEST.COM (short username kudu)
I20260811 19:00:38.436563 18920 webserver.cc:460] Webserver started at http://127.29.112.129:46595/ using document root <none> and password file <none>
I20260811 19:00:38.439920 18920 fs_manager.cc:359] Metadata directory not provided
I20260811 19:00:38.440244 18920 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:00:38.441126 18920 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 19:00:38 dist-test-slave-h6n0 krb5kdc[18042](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474838, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.129@KRBTEST.COM for HTTP/127.29.112.148@KRBTEST.COM
I20260811 19:00:38.486315 18920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ts-0/data/instance:
uuid: "5db07d6a145442db8d947d20919f8d75"
format_stamp: "Formatted at 2026-08-11 19:00:38 on dist-test-slave-h6n0"
server_key: "ca1ef8539062da308640169385483a43"
server_key_iv: "cc36fc743b8c0a878721f2c003b1bad8"
server_key_version: "kuduclusterkey@0"
I20260811 19:00:38.487488 18920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ts-0/wal/instance:
uuid: "5db07d6a145442db8d947d20919f8d75"
format_stamp: "Formatted at 2026-08-11 19:00:38 on dist-test-slave-h6n0"
server_key: "ca1ef8539062da308640169385483a43"
server_key_iv: "cc36fc743b8c0a878721f2c003b1bad8"
server_key_version: "kuduclusterkey@0"
I20260811 19:00:38.496069 18920 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.006s sys 0.000s
I20260811 19:00:38.520802 18951 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:38.523344 18920 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.004s sys 0.001s
I20260811 19:00:38.523604 18920 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ts-0/wal
uuid: "5db07d6a145442db8d947d20919f8d75"
format_stamp: "Formatted at 2026-08-11 19:00:38 on dist-test-slave-h6n0"
server_key: "ca1ef8539062da308640169385483a43"
server_key_iv: "cc36fc743b8c0a878721f2c003b1bad8"
server_key_version: "kuduclusterkey@0"
I20260811 19:00:38.523929 18920 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:00:38.570220 18920 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:00:38.579403 18920 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:00:38.579782 18920 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:00:38.581040 18920 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:00:38.583211 18920 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:00:38.583376 18920 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:38.583546 18920 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:00:38.583649 18920 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:38.650629 18920 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.129:44275
I20260811 19:00:38.650658 19102 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.129:44275 every 8 connection(s)
I20260811 19:00:38.654495 18920 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ts-0/data/info.pb
I20260811 19:00:38.658003 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 18920
I20260811 19:00:38.658324 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ts-0/wal/instance
I20260811 19:00:38.659664 30146 external_mini_cluster.cc:1428] Setting key e034d279ba48f01aac6a3cb9af621069
Aug 11 19:00:38 dist-test-slave-h6n0 krb5kdc[18042](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474838, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.129@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.29.112.130@KRBTEST.COM; defaulting to no policy
I20260811 19:00:38.719496 19108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:38.674472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.129:41179 (local address 127.29.112.190:36797)
0811 19:00:38.683464 (+ 8992us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:38.683477 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:38.685531 (+ 2054us) server_negotiation.cc:407] Connection header received
0811 19:00:38.689795 (+ 4264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:38.689801 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:38.691005 (+ 1204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:38.691482 (+ 477us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:38.695154 (+ 3672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:38.696606 (+ 1452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:38.699569 (+ 2963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:38.700384 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:38.700512 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:38.707928 (+ 7416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:38.707940 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:38.707950 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:38.708025 (+ 75us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:38.713322 (+ 5297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:38.714725 (+ 1403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:38.714733 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:38.714737 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:38.714945 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:38.715724 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:38.715730 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:38.715734 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:38.716316 (+ 582us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:38.716554 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:38.718720 (+ 2166us) server_negotiation.cc:299] Negotiation successful
0811 19:00:38.718999 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8596,"thread_start_us":8448,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.130@KRBTEST.COM" created.
I20260811 19:00:38.721056 19106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:38.673488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:36797 (local address 127.29.112.129:41179)
0811 19:00:38.684298 (+ 10810us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:38.684379 (+ 81us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:38.689315 (+ 4936us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:38.692813 (+ 3498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:38.692830 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:38.693664 (+ 834us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:38.694844 (+ 1180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:38.694879 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:38.696860 (+ 1981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:38.696867 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:38.699337 (+ 2470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:38.699360 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:38.701170 (+ 1810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:38.701244 (+ 74us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:38.701805 (+ 561us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:38.703302 (+ 1497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:38.703351 (+ 49us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:38.707514 (+ 4163us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:00:38.713605 (+ 6091us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:38.713620 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:38.713646 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:38.714513 (+ 867us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:38.715153 (+ 640us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:38.715158 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:38.715162 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:38.715530 (+ 368us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:38.716711 (+ 1181us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:00:38.716721 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:00:38.717784 (+ 1063us) client_negotiation.cc:769] Sending connection context
0811 19:00:38.718602 (+ 818us) client_negotiation.cc:240] Negotiation successful
0811 19:00:38.719230 (+ 628us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10425,"thread_start_us":10133,"threads_started":1}
I20260811 19:00:38.723203 19103 heartbeater.cc:347] Connected to a master server at 127.29.112.190:36797
I20260811 19:00:38.723701 19103 heartbeater.cc:464] Registering TS with master...
I20260811 19:00:38.724808 19103 heartbeater.cc:511] Master 127.29.112.190:36797 requested a full tablet report, sending...
I20260811 19:00:38.728103 18791 ts_manager.cc:194] Registered new tserver with Master: 5db07d6a145442db8d947d20919f8d75 (127.29.112.129:44275)
I20260811 19:00:38.732904 18791 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.129@KRBTEST.COM'} at 127.29.112.129:41179
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.130.keytab.
Entry for principal kudu/127.29.112.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.130.keytab.
W20260811 19:00:38.779609 30146 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.29.112.130: real 0.116s user 0.000s sys 0.035s
I20260811 19:00:38.780596 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.112.148:48391/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.130:0
--local_ip_for_outbound_sockets=127.29.112.130
--webserver_interface=127.29.112.130
--webserver_port=0
--tserver_master_addrs=127.29.112.190:36797
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.130.keytab
--principal=kudu/127.29.112.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:42911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 19:00:38 dist-test-slave-h6n0 krb5kdc[18042](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474838, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.190@KRBTEST.COM for HTTP/127.29.112.148@KRBTEST.COM
Aug 11 19:00:39 dist-test-slave-h6n0 krb5kdc[18042](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474838, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.190@KRBTEST.COM for HTTP/127.29.112.148@KRBTEST.COM
W20260811 19:00:39.402185 19113 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:00:39.403100 19113 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:00:39.403359 19113 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:00:39.417870 19113 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:00:39.418299 19113 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:00:39.418576 19113 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.130
I20260811 19:00:39.438047 19113 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:42911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.29.112.148:48391/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.130.keytab
--principal=kudu/127.29.112.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.112.130
--webserver_port=0
--tserver_master_addrs=127.29.112.190:36797
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.130
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:00:39.440222 19113 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:00:39.442072 19113 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:00:39.480573 19121 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:39.481559 19122 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:39.482194 19126 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:39.487001 19113 server_base.cc:1034] running on GCE node
I20260811 19:00:39.488776 19113 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:00:39.499002 19113 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:00:39.506850 19113 hybrid_clock.cc:648] HybridClock initialized: now 1786474839506669 us; error 217 us; skew 500 ppm
I20260811 19:00:39.507498 19113 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:00:39 dist-test-slave-h6n0 krb5kdc[18042](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474839, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:00:39.513298 19113 init.cc:377] Logged in from keytab as kudu/127.29.112.130@KRBTEST.COM (short username kudu)
I20260811 19:00:39.518147 19113 webserver.cc:460] Webserver started at http://127.29.112.130:42985/ using document root <none> and password file <none>
I20260811 19:00:39.519063 19113 fs_manager.cc:359] Metadata directory not provided
I20260811 19:00:39.519239 19113 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:00:39.519613 19113 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 19:00:39 dist-test-slave-h6n0 krb5kdc[18042](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474839, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.130@KRBTEST.COM for HTTP/127.29.112.148@KRBTEST.COM
I20260811 19:00:39.552093 19113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ts-1/data/instance:
uuid: "9d2ed4daaa6b488b9bc634929daebf5c"
format_stamp: "Formatted at 2026-08-11 19:00:39 on dist-test-slave-h6n0"
server_key: "958ab240797db127dde07e97126e6411"
server_key_iv: "3c7aaf5d0c09c8d55eb1f763dadf4d44"
server_key_version: "kuduclusterkey@0"
I20260811 19:00:39.553591 19113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ts-1/wal/instance:
uuid: "9d2ed4daaa6b488b9bc634929daebf5c"
format_stamp: "Formatted at 2026-08-11 19:00:39 on dist-test-slave-h6n0"
server_key: "958ab240797db127dde07e97126e6411"
server_key_iv: "3c7aaf5d0c09c8d55eb1f763dadf4d44"
server_key_version: "kuduclusterkey@0"
I20260811 19:00:39.567034 19113 fs_manager.cc:691] Time spent creating directory manager: real 0.013s user 0.008s sys 0.004s
I20260811 19:00:39.594776 19146 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:39.596998 19113 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.007s sys 0.000s
I20260811 19:00:39.597221 19113 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ts-1/wal
uuid: "9d2ed4daaa6b488b9bc634929daebf5c"
format_stamp: "Formatted at 2026-08-11 19:00:39 on dist-test-slave-h6n0"
server_key: "958ab240797db127dde07e97126e6411"
server_key_iv: "3c7aaf5d0c09c8d55eb1f763dadf4d44"
server_key_version: "kuduclusterkey@0"
I20260811 19:00:39.597507 19113 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:00:39.642619 19113 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:00:39.648504 19113 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:00:39.648934 19113 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:00:39.657688 19113 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:00:39.663479 19113 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:00:39.663641 19113 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:39.663787 19113 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:00:39.663853 19113 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:39.716032 19113 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.130:33745
I20260811 19:00:39.716068 19307 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.130:33745 every 8 connection(s)
I20260811 19:00:39.718106 19113 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ts-1/data/info.pb
I20260811 19:00:39.726339 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 19113
I20260811 19:00:39.727017 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ts-1/wal/instance
I20260811 19:00:39.727663 30146 external_mini_cluster.cc:1428] Setting key bfa0986a53579b0df7ca54bd38444e3b
I20260811 19:00:39.740657 19103 heartbeater.cc:503] Master 127.29.112.190:36797 was elected leader, sending a full tablet report...
Aug 11 19:00:39 dist-test-slave-h6n0 krb5kdc[18042](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474839, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.130@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 19:00:39.762931 19310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:39.721869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:36797 (local address 127.29.112.130:60827)
0811 19:00:39.724626 (+ 2757us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:39.724715 (+ 89us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:39.728110 (+ 3395us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:39.737902 (+ 9792us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:39.737917 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:39.738985 (+ 1068us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:39.739944 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:39.739974 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:39.742100 (+ 2126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:39.742107 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:39.744518 (+ 2411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:39.744544 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:39.745995 (+ 1451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:39.746003 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:39.746307 (+ 304us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:39.747275 (+ 968us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:39.747308 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:39.750119 (+ 2811us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:00:39.755886 (+ 5767us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:39.755904 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:39.755933 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:39.756863 (+ 930us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:39.757525 (+ 662us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:39.757530 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:39.757534 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:39.757866 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:39.759237 (+ 1371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:00:39.759250 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:00:39.759850 (+ 600us) client_negotiation.cc:769] Sending connection context
0811 19:00:39.760247 (+ 397us) client_negotiation.cc:240] Negotiation successful
0811 19:00:39.760897 (+ 650us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2273,"thread_start_us":1877,"threads_started":1}
I20260811 19:00:39.764976 19312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:39.722620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.130:60827 (local address 127.29.112.190:36797)
0811 19:00:39.736983 (+ 14363us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:39.736990 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:39.737018 (+ 28us) server_negotiation.cc:407] Connection header received
0811 19:00:39.737088 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:39.737094 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:39.737205 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:39.740332 (+ 3127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:39.740542 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:39.741727 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:39.744788 (+ 3061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:39.745646 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:39.745822 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:39.750468 (+ 4646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:39.750479 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:39.750488 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:39.750590 (+ 102us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:39.755419 (+ 4829us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:39.757120 (+ 1701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:39.757128 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:39.757131 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:39.757350 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:39.758025 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:39.758030 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:39.758033 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:39.758845 (+ 812us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:39.759084 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:39.764046 (+ 4962us) server_negotiation.cc:299] Negotiation successful
0811 19:00:39.764476 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14211,"thread_start_us":14712,"threads_started":1}
I20260811 19:00:39.768185 19308 heartbeater.cc:347] Connected to a master server at 127.29.112.190:36797
I20260811 19:00:39.768683 19308 heartbeater.cc:464] Registering TS with master...
I20260811 19:00:39.769941 19308 heartbeater.cc:511] Master 127.29.112.190:36797 requested a full tablet report, sending...
I20260811 19:00:39.772473 18791 ts_manager.cc:194] Registered new tserver with Master: 9d2ed4daaa6b488b9bc634929daebf5c (127.29.112.130:33745)
I20260811 19:00:39.773856 18791 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.130@KRBTEST.COM'} at 127.29.112.130:60827
WARNING: no policy specified for kudu/127.29.112.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.131.keytab.
Entry for principal kudu/127.29.112.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.131.keytab.
I20260811 19:00:39.836402 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.112.148:48391/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.131:0
--local_ip_for_outbound_sockets=127.29.112.131
--webserver_interface=127.29.112.131
--webserver_port=0
--tserver_master_addrs=127.29.112.190:36797
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.131.keytab
--principal=kudu/127.29.112.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:42911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:00:40.307114 19316 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:00:40.307483 19316 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:00:40.307675 19316 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:00:40.315502 19316 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:00:40.315667 19316 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:00:40.315850 19316 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.131
I20260811 19:00:40.326085 19316 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:42911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.29.112.148:48391/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.131.keytab
--principal=kudu/127.29.112.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.112.131
--webserver_port=0
--tserver_master_addrs=127.29.112.190:36797
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.131
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:00:40.327728 19316 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:00:40.329504 19316 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:00:40.356269 19321 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:40.357276 19323 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:40.362999 19316 server_base.cc:1034] running on GCE node
W20260811 19:00:40.364233 19326 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:40.365479 19316 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:00:40.367139 19316 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:00:40.368428 19316 hybrid_clock.cc:648] HybridClock initialized: now 1786474840368381 us; error 44 us; skew 500 ppm
I20260811 19:00:40.368927 19316 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:00:40 dist-test-slave-h6n0 krb5kdc[18042](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474840, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:00:40.372754 19316 init.cc:377] Logged in from keytab as kudu/127.29.112.131@KRBTEST.COM (short username kudu)
I20260811 19:00:40.375403 19316 webserver.cc:460] Webserver started at http://127.29.112.131:43409/ using document root <none> and password file <none>
I20260811 19:00:40.376312 19316 fs_manager.cc:359] Metadata directory not provided
I20260811 19:00:40.376477 19316 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:00:40.377540 19316 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 19:00:40 dist-test-slave-h6n0 krb5kdc[18042](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474840, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.131@KRBTEST.COM for HTTP/127.29.112.148@KRBTEST.COM
I20260811 19:00:40.398869 19316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ts-2/data/instance:
uuid: "2a3dc828d7cb4336841a377a442498b0"
format_stamp: "Formatted at 2026-08-11 19:00:40 on dist-test-slave-h6n0"
server_key: "402c4c24b31e9c1174610f90333e905f"
server_key_iv: "0a7a1624b0d2cfd94ba375ae045b171d"
server_key_version: "kuduclusterkey@0"
I20260811 19:00:40.399941 19316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ts-2/wal/instance:
uuid: "2a3dc828d7cb4336841a377a442498b0"
format_stamp: "Formatted at 2026-08-11 19:00:40 on dist-test-slave-h6n0"
server_key: "402c4c24b31e9c1174610f90333e905f"
server_key_iv: "0a7a1624b0d2cfd94ba375ae045b171d"
server_key_version: "kuduclusterkey@0"
I20260811 19:00:40.410976 19316 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.007s sys 0.003s
I20260811 19:00:40.437722 19334 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:40.440721 19316 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.002s sys 0.001s
I20260811 19:00:40.441099 19316 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ts-2/wal
uuid: "2a3dc828d7cb4336841a377a442498b0"
format_stamp: "Formatted at 2026-08-11 19:00:40 on dist-test-slave-h6n0"
server_key: "402c4c24b31e9c1174610f90333e905f"
server_key_iv: "0a7a1624b0d2cfd94ba375ae045b171d"
server_key_version: "kuduclusterkey@0"
I20260811 19:00:40.441413 19316 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:00:40.465389 19316 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:00:40.470513 19316 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:00:40.470945 19316 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:00:40.472103 19316 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:00:40.475135 19316 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:00:40.475402 19316 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:40.475699 19316 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:00:40.475766 19316 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:40.540975 19316 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.131:38147
I20260811 19:00:40.541021 19519 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.131:38147 every 8 connection(s)
I20260811 19:00:40.542434 19316 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ts-2/data/info.pb
I20260811 19:00:40.547838 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 19316
I20260811 19:00:40.548032 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ts-2/wal/instance
I20260811 19:00:40.548732 30146 external_mini_cluster.cc:1428] Setting key 6a06660e9934b63b5e4b25ba1914ba75
Aug 11 19:00:40 dist-test-slave-h6n0 krb5kdc[18042](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474840, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.131@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 19:00:40.573019 19522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:40.545872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.131:51501 (local address 127.29.112.190:36797)
0811 19:00:40.546637 (+ 765us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:40.546647 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:40.548323 (+ 1676us) server_negotiation.cc:407] Connection header received
0811 19:00:40.551915 (+ 3592us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:40.551922 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:40.552072 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:40.552246 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:40.554211 (+ 1965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:40.555482 (+ 1271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:40.557306 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:40.557973 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:40.558103 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:40.564003 (+ 5900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:40.564010 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:40.564014 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:40.564062 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:40.569110 (+ 5048us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:40.570058 (+ 948us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:40.570063 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:40.570066 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:40.570246 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:40.570897 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:40.570902 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:40.570904 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:40.571426 (+ 522us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:40.571623 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:40.572530 (+ 907us) server_negotiation.cc:299] Negotiation successful
0811 19:00:40.572749 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":569,"thread_start_us":396,"threads_started":1}
I20260811 19:00:40.574123 19526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:40.546618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:36797 (local address 127.29.112.131:51501)
0811 19:00:40.548042 (+ 1424us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:40.548138 (+ 96us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:40.551413 (+ 3275us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:40.552607 (+ 1194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:40.552618 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:40.553287 (+ 669us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:40.553963 (+ 676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:40.553982 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:40.555700 (+ 1718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:40.555704 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:40.557093 (+ 1389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:40.557107 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:40.558254 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:40.558260 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:40.558607 (+ 347us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:40.559829 (+ 1222us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:40.559858 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:40.563530 (+ 3672us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:00:40.569321 (+ 5791us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:40.569330 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:40.569346 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:40.569875 (+ 529us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:40.570383 (+ 508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:40.570386 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:40.570388 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:40.570716 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:40.571660 (+ 944us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:00:40.571667 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:00:40.572077 (+ 410us) client_negotiation.cc:769] Sending connection context
0811 19:00:40.572349 (+ 272us) client_negotiation.cc:240] Negotiation successful
0811 19:00:40.572788 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":991,"thread_start_us":689,"threads_started":1}
I20260811 19:00:40.576481 19520 heartbeater.cc:347] Connected to a master server at 127.29.112.190:36797
I20260811 19:00:40.576858 19520 heartbeater.cc:464] Registering TS with master...
I20260811 19:00:40.577828 19520 heartbeater.cc:511] Master 127.29.112.190:36797 requested a full tablet report, sending...
I20260811 19:00:40.580111 18791 ts_manager.cc:194] Registered new tserver with Master: 2a3dc828d7cb4336841a377a442498b0 (127.29.112.131:38147)
I20260811 19:00:40.581208 18791 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.131@KRBTEST.COM'} at 127.29.112.131:51501
I20260811 19:00:40.582435 30146 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 19:00:40.585196 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 18920
Aug 11 19:00:40 dist-test-slave-h6n0 krb5kdc[18042](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474835, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.129@KRBTEST.COM
I20260811 19:00:40.609048 19531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:40.586688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54356 (local address 127.29.112.129:44275)
0811 19:00:40.587385 (+ 697us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:40.587392 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:40.587425 (+ 33us) server_negotiation.cc:407] Connection header received
0811 19:00:40.587616 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:40.587623 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:40.587845 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:40.588006 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:40.589804 (+ 1798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:40.591101 (+ 1297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:40.593013 (+ 1912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:40.594060 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:40.594516 (+ 456us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:40.601019 (+ 6503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:40.601038 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:40.601056 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:40.601110 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:40.604865 (+ 3755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:40.605881 (+ 1016us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:40.605894 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:40.605897 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:40.606310 (+ 413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:40.606916 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:40.606920 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:40.606923 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:40.607457 (+ 534us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:40.607847 (+ 390us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:40.608338 (+ 491us) server_negotiation.cc:299] Negotiation successful
0811 19:00:40.608665 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":525,"thread_start_us":383,"threads_started":1}
I20260811 19:00:40.611105 19098 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:00:40.712944 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 18920
I20260811 19:00:40.744287 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 19113
Aug 11 19:00:40 dist-test-slave-h6n0 krb5kdc[18042](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474835, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.130@KRBTEST.COM
I20260811 19:00:40.771692 19533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:40.752026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52840 (local address 127.29.112.130:33745)
0811 19:00:40.752674 (+ 648us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:40.752682 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:40.752709 (+ 27us) server_negotiation.cc:407] Connection header received
0811 19:00:40.752773 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:40.752780 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:40.752998 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:40.753130 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:40.754874 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:40.756227 (+ 1353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:40.757674 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:40.758302 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:40.758406 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:40.764476 (+ 6070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:40.764497 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:40.764518 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:40.764580 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:40.768303 (+ 3723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:40.769149 (+ 846us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:40.769158 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:40.769161 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:40.769355 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:40.769850 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:40.769854 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:40.769856 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:40.770291 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:40.770679 (+ 388us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:40.771013 (+ 334us) server_negotiation.cc:299] Negotiation successful
0811 19:00:40.771293 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":478,"thread_start_us":323,"threads_started":1}
I20260811 19:00:40.774056 19303 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:00:40.778352 19308 heartbeater.cc:503] Master 127.29.112.190:36797 was elected leader, sending a full tablet report...
I20260811 19:00:40.865708 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 19113
I20260811 19:00:40.901764 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 19316
Aug 11 19:00:40 dist-test-slave-h6n0 krb5kdc[18042](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474835, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.131@KRBTEST.COM
I20260811 19:00:40.926684 19536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:40.903591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35464 (local address 127.29.112.131:38147)
0811 19:00:40.904612 (+ 1021us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:40.904620 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:40.904659 (+ 39us) server_negotiation.cc:407] Connection header received
0811 19:00:40.904732 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:40.904740 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:40.905044 (+ 304us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:40.905216 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:40.907637 (+ 2421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:40.909205 (+ 1568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:40.911183 (+ 1978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:40.912044 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:40.912161 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:40.918238 (+ 6077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:40.918252 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:40.918269 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:40.918316 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:40.922443 (+ 4127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:40.923527 (+ 1084us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:40.923535 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:40.923538 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:40.923714 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:40.924267 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:40.924271 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:40.924273 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:40.924682 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:40.924977 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:40.925680 (+ 703us) server_negotiation.cc:299] Negotiation successful
0811 19:00:40.925992 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":831,"thread_start_us":634,"threads_started":1}
I20260811 19:00:40.928121 19514 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:00:41.025401 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 19316
I20260811 19:00:41.054759 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 18746
I20260811 19:00:41.056694 18844 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:00:41.181910 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 18746
I20260811 19:00:41.219911 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.190:36797
--webserver_interface=127.29.112.190
--webserver_port=36799
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab
--principal=kudu/127.29.112.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:42911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.190:36797
--ranger_config_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.112.148:48391/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.29.112.148:48391/kms with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:00:41.600608 19540 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:00:41.601035 19540 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:00:41.601251 19540 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:00:41.610050 19540 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:00:41.610241 19540 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:00:41.610369 19540 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:00:41.610481 19540 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:00:41.610641 19540 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:00:41.622411 19540 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:42911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.29.112.148:48391/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.29.112.190:36797
--ranger_config_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab
--principal=kudu/127.29.112.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.190:36797
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.112.190
--webserver_port=36799
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:00:41.623915 19540 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:00:41.625934 19540 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:00:41.641048 19546 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:41.641126 19551 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:41.641126 19547 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:41.642293 19540 server_base.cc:1034] running on GCE node
I20260811 19:00:41.643257 19540 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:00:41.644824 19540 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:00:41.646328 19540 hybrid_clock.cc:648] HybridClock initialized: now 1786474841646194 us; error 54 us; skew 500 ppm
I20260811 19:00:41.646934 19540 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:00:41 dist-test-slave-h6n0 krb5kdc[18042](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474841, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:00:41.650686 19540 init.cc:377] Logged in from keytab as kudu/127.29.112.190@KRBTEST.COM (short username kudu)
I20260811 19:00:41.652604 19540 webserver.cc:460] Webserver started at http://127.29.112.190:36799/ using document root <none> and password file <none>
I20260811 19:00:41.653373 19540 fs_manager.cc:359] Metadata directory not provided
I20260811 19:00:41.653503 19540 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 19:00:41 dist-test-slave-h6n0 krb5kdc[18042](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474841, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.190@KRBTEST.COM for HTTP/127.29.112.148@KRBTEST.COM
I20260811 19:00:41.746876 19540 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.001s sys 0.004s
I20260811 19:00:41.750095 19567 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:41.751991 19540 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 19:00:41.752224 19540 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/master-0/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/master-0/wal
uuid: "daba4ff62b2d4177855a7a8bc0ed5bde"
format_stamp: "Formatted at 2026-08-11 19:00:36 on dist-test-slave-h6n0"
server_key: "04d8d55189a02f6212c8a1767fd8bfdd"
server_key_iv: "480e2d1f2fba0f32ebc52ffb1644e2d2"
server_key_version: "kuduclusterkey@0"
I20260811 19:00:41.753185 19540 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:00:41.772118 19540 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:00:41.776458 19540 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:00:41.776784 19540 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:00:41.804200 19653 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.190:36797 every 8 connection(s)
I20260811 19:00:41.804241 19540 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.190:36797
I20260811 19:00:41.807624 19540 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
I20260811 19:00:41.817317 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 19540
I20260811 19:00:41.818218 19654 sys_catalog.cc:263] Verifying existing consensus state
I20260811 19:00:41.818406 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.112.148:48391/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.129:44275
--local_ip_for_outbound_sockets=127.29.112.129
--tserver_master_addrs=127.29.112.190:36797
--webserver_port=46595
--webserver_interface=127.29.112.129
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.129.keytab
--principal=kudu/127.29.112.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:42911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.29.112.148:48391/kms with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 19:00:41.822327 19654 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P daba4ff62b2d4177855a7a8bc0ed5bde: Bootstrap starting.
I20260811 19:00:41.847757 19654 log.cc:824] T 00000000000000000000000000000000 P daba4ff62b2d4177855a7a8bc0ed5bde: Log is configured to *not* fsync() on all Append() calls
I20260811 19:00:41.859130 19654 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P daba4ff62b2d4177855a7a8bc0ed5bde: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 19:00:41.859782 19654 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P daba4ff62b2d4177855a7a8bc0ed5bde: Bootstrap complete.
I20260811 19:00:41.870839 19654 raft_consensus.cc:348] T 00000000000000000000000000000000 P daba4ff62b2d4177855a7a8bc0ed5bde [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "daba4ff62b2d4177855a7a8bc0ed5bde" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 36797 } }
I20260811 19:00:41.871882 19654 raft_consensus.cc:729] T 00000000000000000000000000000000 P daba4ff62b2d4177855a7a8bc0ed5bde [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: daba4ff62b2d4177855a7a8bc0ed5bde, State: Initialized, Role: FOLLOWER
I20260811 19:00:41.872677 19654 consensus_queue.cc:260] T 00000000000000000000000000000000 P daba4ff62b2d4177855a7a8bc0ed5bde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "daba4ff62b2d4177855a7a8bc0ed5bde" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 36797 } }
I20260811 19:00:41.873214 19654 raft_consensus.cc:388] T 00000000000000000000000000000000 P daba4ff62b2d4177855a7a8bc0ed5bde [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:00:41.873370 19654 raft_consensus.cc:482] T 00000000000000000000000000000000 P daba4ff62b2d4177855a7a8bc0ed5bde [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:00:41.873603 19654 raft_consensus.cc:3037] T 00000000000000000000000000000000 P daba4ff62b2d4177855a7a8bc0ed5bde [term 1 FOLLOWER]: Advancing to term 2
I20260811 19:00:41.879076 19654 raft_consensus.cc:504] T 00000000000000000000000000000000 P daba4ff62b2d4177855a7a8bc0ed5bde [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "daba4ff62b2d4177855a7a8bc0ed5bde" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 36797 } }
I20260811 19:00:41.879689 19654 leader_election.cc:302] T 00000000000000000000000000000000 P daba4ff62b2d4177855a7a8bc0ed5bde [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: daba4ff62b2d4177855a7a8bc0ed5bde; no voters:
I20260811 19:00:41.880427 19654 leader_election.cc:288] T 00000000000000000000000000000000 P daba4ff62b2d4177855a7a8bc0ed5bde [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 19:00:41.880591 19658 raft_consensus.cc:2781] T 00000000000000000000000000000000 P daba4ff62b2d4177855a7a8bc0ed5bde [term 2 FOLLOWER]: Leader election won for term 2
I20260811 19:00:41.882079 19658 raft_consensus.cc:686] T 00000000000000000000000000000000 P daba4ff62b2d4177855a7a8bc0ed5bde [term 2 LEADER]: Becoming Leader. State: Replica: daba4ff62b2d4177855a7a8bc0ed5bde, State: Running, Role: LEADER
I20260811 19:00:41.882860 19658 consensus_queue.cc:237] T 00000000000000000000000000000000 P daba4ff62b2d4177855a7a8bc0ed5bde [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "daba4ff62b2d4177855a7a8bc0ed5bde" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 36797 } }
I20260811 19:00:41.883132 19654 sys_catalog.cc:564] T 00000000000000000000000000000000 P daba4ff62b2d4177855a7a8bc0ed5bde [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:00:41.886420 19662 sys_catalog.cc:455] T 00000000000000000000000000000000 P daba4ff62b2d4177855a7a8bc0ed5bde [sys.catalog]: SysCatalogTable state changed. Reason: New leader daba4ff62b2d4177855a7a8bc0ed5bde. Latest consensus state: current_term: 2 leader_uuid: "daba4ff62b2d4177855a7a8bc0ed5bde" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "daba4ff62b2d4177855a7a8bc0ed5bde" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 36797 } } }
I20260811 19:00:41.887137 19662 sys_catalog.cc:458] T 00000000000000000000000000000000 P daba4ff62b2d4177855a7a8bc0ed5bde [sys.catalog]: This master's current role is: LEADER
I20260811 19:00:41.888036 19661 sys_catalog.cc:455] T 00000000000000000000000000000000 P daba4ff62b2d4177855a7a8bc0ed5bde [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "daba4ff62b2d4177855a7a8bc0ed5bde" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "daba4ff62b2d4177855a7a8bc0ed5bde" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 36797 } } }
I20260811 19:00:41.888677 19661 sys_catalog.cc:458] T 00000000000000000000000000000000 P daba4ff62b2d4177855a7a8bc0ed5bde [sys.catalog]: This master's current role is: LEADER
I20260811 19:00:41.898092 19671 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:00:41.905870 19671 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:00:41.907429 19671 catalog_manager.cc:1259] Loaded cluster ID: 763fcf9fa3bb4ee7b3dbce487ec850bf
I20260811 19:00:41.907610 19671 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:00:41.911659 19671 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:00:41.913139 19671 catalog_manager.cc:5879] T 00000000000000000000000000000000 P daba4ff62b2d4177855a7a8bc0ed5bde: Loaded TSK: 0
I20260811 19:00:41.914877 19654 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 19:00:41.915071 19671 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 19:00:42.341084 19656 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:00:42.341470 19656 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:00:42.341646 19656 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:00:42.370779 19656 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:00:42.371088 19656 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:00:42.371634 19656 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.129
I20260811 19:00:42.390398 19656 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:42911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.29.112.148:48391/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.129.keytab
--principal=kudu/127.29.112.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.129:44275
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.112.129
--webserver_port=46595
--tserver_master_addrs=127.29.112.190:36797
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.129
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:00:42.392676 19656 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:00:42.394761 19656 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:00:42.431581 19721 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:42.433073 19723 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:42.434736 19726 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:42.440601 19656 server_base.cc:1034] running on GCE node
I20260811 19:00:42.441969 19656 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:00:42.444924 19656 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:00:42.452646 19656 hybrid_clock.cc:648] HybridClock initialized: now 1786474842451171 us; error 1451 us; skew 500 ppm
I20260811 19:00:42.453241 19656 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:00:42 dist-test-slave-h6n0 krb5kdc[18042](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474842, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:00:42.464401 19656 init.cc:377] Logged in from keytab as kudu/127.29.112.129@KRBTEST.COM (short username kudu)
I20260811 19:00:42.466907 19656 webserver.cc:460] Webserver started at http://127.29.112.129:46595/ using document root <none> and password file <none>
I20260811 19:00:42.467996 19656 fs_manager.cc:359] Metadata directory not provided
I20260811 19:00:42.468253 19656 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 19:00:42 dist-test-slave-h6n0 krb5kdc[18042](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474842, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.129@KRBTEST.COM for HTTP/127.29.112.148@KRBTEST.COM
I20260811 19:00:42.524644 19656 fs_manager.cc:709] Time spent opening directory manager: real 0.012s user 0.005s sys 0.001s
I20260811 19:00:42.547499 19738 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:42.552399 19656 fs_manager.cc:725] Time spent opening block manager: real 0.025s user 0.005s sys 0.015s
I20260811 19:00:42.553010 19656 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ts-0/wal
uuid: "5db07d6a145442db8d947d20919f8d75"
format_stamp: "Formatted at 2026-08-11 19:00:38 on dist-test-slave-h6n0"
server_key: "ca1ef8539062da308640169385483a43"
server_key_iv: "cc36fc743b8c0a878721f2c003b1bad8"
server_key_version: "kuduclusterkey@0"
I20260811 19:00:42.554790 19656 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:00:42.592820 19656 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:00:42.600186 19656 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:00:42.600615 19656 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:00:42.602267 19656 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:00:42.604810 19656 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:00:42.604954 19656 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:42.605096 19656 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:00:42.605180 19656 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:42.705994 19656 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.129:44275
I20260811 19:00:42.706064 19926 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.129:44275 every 8 connection(s)
I20260811 19:00:42.708257 19656 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ts-0/data/info.pb
I20260811 19:00:42.718945 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 19656
I20260811 19:00:42.720219 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.112.148:48391/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.130:33745
--local_ip_for_outbound_sockets=127.29.112.130
--tserver_master_addrs=127.29.112.190:36797
--webserver_port=42985
--webserver_interface=127.29.112.130
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.130.keytab
--principal=kudu/127.29.112.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:42911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.29.112.148:48391/kms with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 19:00:42 dist-test-slave-h6n0 krb5kdc[18042](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474842, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.129@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 19:00:42.751048 19932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:42.713778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:36797 (local address 127.29.112.129:51999)
0811 19:00:42.715470 (+ 1692us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:42.715529 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:42.719208 (+ 3679us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:42.722152 (+ 2944us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:42.722167 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:42.723150 (+ 983us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:42.724130 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:42.724160 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:42.727440 (+ 3280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:42.727451 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:42.730075 (+ 2624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:42.730103 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:42.732062 (+ 1959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:42.732073 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:42.733201 (+ 1128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:42.734226 (+ 1025us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:42.734273 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:42.739468 (+ 5195us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:00:42.745475 (+ 6007us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:42.745487 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:42.745507 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:42.746223 (+ 716us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:42.746876 (+ 653us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:42.746881 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:42.746886 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:42.747113 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:42.748146 (+ 1033us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:00:42.748155 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:00:42.748525 (+ 370us) client_negotiation.cc:769] Sending connection context
0811 19:00:42.748839 (+ 314us) client_negotiation.cc:240] Negotiation successful
0811 19:00:42.749489 (+ 650us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1362,"thread_start_us":449,"threads_started":1}
I20260811 19:00:42.751048 19931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:42.713758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.129:51999 (local address 127.29.112.190:36797)
0811 19:00:42.715139 (+ 1381us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:42.715162 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:42.715823 (+ 661us) server_negotiation.cc:407] Connection header received
0811 19:00:42.720492 (+ 4669us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:42.720526 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:42.721268 (+ 742us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:42.721886 (+ 618us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:42.724517 (+ 2631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:42.726799 (+ 2282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:42.730390 (+ 3591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:42.731700 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:42.732034 (+ 334us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:42.739923 (+ 7889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:42.739937 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:42.739959 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:42.740024 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:42.745195 (+ 5171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:42.746443 (+ 1248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:42.746451 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:42.746457 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:42.746724 (+ 267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:42.747301 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:42.747307 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:42.747310 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:42.747942 (+ 632us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:42.748209 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:42.749068 (+ 859us) server_negotiation.cc:299] Negotiation successful
0811 19:00:42.749489 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1084,"thread_start_us":806,"threads_started":1}
I20260811 19:00:42.755235 19929 heartbeater.cc:347] Connected to a master server at 127.29.112.190:36797
I20260811 19:00:42.755597 19929 heartbeater.cc:464] Registering TS with master...
I20260811 19:00:42.756561 19929 heartbeater.cc:511] Master 127.29.112.190:36797 requested a full tablet report, sending...
W20260811 19:00:42.760010 19929 heartbeater.cc:644] Failed to heartbeat to 127.29.112.190:36797 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 19:00:42.760272 19929 heartbeater.cc:464] Registering TS with master...
I20260811 19:00:42.760555 19929 heartbeater.cc:511] Master 127.29.112.190:36797 requested a full tablet report, sending...
I20260811 19:00:42.761713 19929 heartbeater.cc:464] Registering TS with master...
I20260811 19:00:42.762089 19929 heartbeater.cc:511] Master 127.29.112.190:36797 requested a full tablet report, sending...
W20260811 19:00:42.763275 19929 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 19:00:43.244287 19933 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:00:43.244752 19933 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:00:43.244971 19933 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:00:43.255450 19933 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:00:43.255651 19933 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:00:43.255865 19933 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.130
I20260811 19:00:43.270987 19933 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:42911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.29.112.148:48391/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.130.keytab
--principal=kudu/127.29.112.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.130:33745
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.112.130
--webserver_port=42985
--tserver_master_addrs=127.29.112.190:36797
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.130
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:00:43.272878 19933 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:00:43.274382 19933 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:00:43.293524 19960 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:43.293867 19957 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:43.294129 19955 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:43.295315 19933 server_base.cc:1034] running on GCE node
I20260811 19:00:43.296200 19933 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:00:43.305671 19933 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:00:43.307125 19933 hybrid_clock.cc:648] HybridClock initialized: now 1786474843307040 us; error 80 us; skew 500 ppm
I20260811 19:00:43.307605 19933 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:00:43 dist-test-slave-h6n0 krb5kdc[18042](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474843, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:00:43.312531 19933 init.cc:377] Logged in from keytab as kudu/127.29.112.130@KRBTEST.COM (short username kudu)
I20260811 19:00:43.315306 19933 webserver.cc:460] Webserver started at http://127.29.112.130:42985/ using document root <none> and password file <none>
I20260811 19:00:43.316411 19933 fs_manager.cc:359] Metadata directory not provided
I20260811 19:00:43.316619 19933 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 19:00:43 dist-test-slave-h6n0 krb5kdc[18042](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474843, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.130@KRBTEST.COM for HTTP/127.29.112.148@KRBTEST.COM
I20260811 19:00:43.360179 19933 fs_manager.cc:709] Time spent opening directory manager: real 0.019s user 0.014s sys 0.004s
I20260811 19:00:43.375797 19970 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:43.377735 19933 fs_manager.cc:725] Time spent opening block manager: real 0.011s user 0.002s sys 0.007s
I20260811 19:00:43.378334 19933 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ts-1/wal
uuid: "9d2ed4daaa6b488b9bc634929daebf5c"
format_stamp: "Formatted at 2026-08-11 19:00:39 on dist-test-slave-h6n0"
server_key: "958ab240797db127dde07e97126e6411"
server_key_iv: "3c7aaf5d0c09c8d55eb1f763dadf4d44"
server_key_version: "kuduclusterkey@0"
I20260811 19:00:43.380203 19933 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:00:43.408500 19933 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:00:43.414060 19933 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:00:43.414445 19933 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:00:43.418282 19933 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:00:43.420572 19933 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:00:43.420709 19933 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:43.420828 19933 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:00:43.420910 19933 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:43.488214 19933 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.130:33745
I20260811 19:00:43.488278 20150 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.130:33745 every 8 connection(s)
I20260811 19:00:43.489851 19933 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ts-1/data/info.pb
I20260811 19:00:43.490475 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 19933
I20260811 19:00:43.491588 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.112.148:48391/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.131:38147
--local_ip_for_outbound_sockets=127.29.112.131
--tserver_master_addrs=127.29.112.190:36797
--webserver_port=43409
--webserver_interface=127.29.112.131
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.131.keytab
--principal=kudu/127.29.112.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:42911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.29.112.148:48391/kms with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 19:00:43 dist-test-slave-h6n0 krb5kdc[18042](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786474843, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 11 19:00:43 dist-test-slave-h6n0 krb5kdc[18042](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474843, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.130@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 19:00:43.558071 20159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:43.514242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.130:42795 (local address 127.29.112.190:36797)
0811 19:00:43.525627 (+ 11385us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:43.525636 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:43.526583 (+ 947us) server_negotiation.cc:407] Connection header received
0811 19:00:43.529558 (+ 2975us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:43.529568 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:43.529708 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:43.529814 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:43.532412 (+ 2598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:43.533755 (+ 1343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:43.536524 (+ 2769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:43.537329 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:43.537447 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:43.548122 (+ 10675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:43.548132 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:43.548137 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:43.548191 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:43.552112 (+ 3921us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:43.553255 (+ 1143us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:43.553261 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:43.553265 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:43.553432 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:43.554000 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:43.554005 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:43.554009 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:43.554372 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:43.554543 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:43.557418 (+ 2875us) server_negotiation.cc:299] Negotiation successful
0811 19:00:43.557679 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11213,"thread_start_us":11052,"threads_started":1}
I20260811 19:00:43.560443 20158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:43.513365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:36797 (local address 127.29.112.130:42795)
0811 19:00:43.526302 (+ 12937us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:43.526364 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:43.529169 (+ 2805us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:43.530445 (+ 1276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:43.530460 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:43.531265 (+ 805us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:43.532187 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:43.532218 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:43.533995 (+ 1777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:43.534003 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:43.536318 (+ 2315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:43.536338 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:43.538169 (+ 1831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:43.538179 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:43.538613 (+ 434us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:43.539593 (+ 980us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:43.539636 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:43.547648 (+ 8012us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:00:43.552342 (+ 4694us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:43.552356 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:43.552381 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:43.553075 (+ 694us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:43.553569 (+ 494us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:43.553574 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:43.553578 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:43.553862 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:43.554620 (+ 758us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:00:43.554629 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:00:43.555100 (+ 471us) client_negotiation.cc:769] Sending connection context
0811 19:00:43.555408 (+ 308us) client_negotiation.cc:240] Negotiation successful
0811 19:00:43.556007 (+ 599us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12529,"thread_start_us":14416,"threads_started":1}
I20260811 19:00:43.563158 20151 heartbeater.cc:347] Connected to a master server at 127.29.112.190:36797
I20260811 19:00:43.563530 20151 heartbeater.cc:464] Registering TS with master...
I20260811 19:00:43.564478 20151 heartbeater.cc:511] Master 127.29.112.190:36797 requested a full tablet report, sending...
I20260811 19:00:43.566828 19598 ts_manager.cc:194] Registered new tserver with Master: 9d2ed4daaa6b488b9bc634929daebf5c (127.29.112.130:33745)
I20260811 19:00:43.569136 19598 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.130@KRBTEST.COM'} at 127.29.112.130:42795
I20260811 19:00:43.763528 19929 heartbeater.cc:464] Registering TS with master...
I20260811 19:00:43.764024 19929 heartbeater.cc:511] Master 127.29.112.190:36797 requested a full tablet report, sending...
I20260811 19:00:43.765473 19598 ts_manager.cc:194] Registered new tserver with Master: 5db07d6a145442db8d947d20919f8d75 (127.29.112.129:44275)
I20260811 19:00:43.766705 19598 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.129@KRBTEST.COM'} at 127.29.112.129:51999
W20260811 19:00:44.007865 20154 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:00:44.008436 20154 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:00:44.008656 20154 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:00:44.023955 20154 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:00:44.024168 20154 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:00:44.024454 20154 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.131
I20260811 19:00:44.041925 20154 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:42911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.29.112.148:48391/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.131.keytab
--principal=kudu/127.29.112.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.131:38147
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.112.131
--webserver_port=43409
--tserver_master_addrs=127.29.112.190:36797
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.131
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:00:44.044103 20154 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:00:44.046060 20154 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:00:44.078338 20171 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:44.078704 20172 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:44.080335 20174 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:44.081809 20154 server_base.cc:1034] running on GCE node
I20260811 19:00:44.082719 20154 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:00:44.084465 20154 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:00:44.085748 20154 hybrid_clock.cc:648] HybridClock initialized: now 1786474844085678 us; error 54 us; skew 500 ppm
I20260811 19:00:44.086158 20154 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:00:44 dist-test-slave-h6n0 krb5kdc[18042](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474843, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.190@KRBTEST.COM for HTTP/127.29.112.148@KRBTEST.COM
Aug 11 19:00:44 dist-test-slave-h6n0 krb5kdc[18042](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474844, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:00:44.091104 20154 init.cc:377] Logged in from keytab as kudu/127.29.112.131@KRBTEST.COM (short username kudu)
I20260811 19:00:44.122510 20154 webserver.cc:460] Webserver started at http://127.29.112.131:43409/ using document root <none> and password file <none>
I20260811 19:00:44.124058 20154 fs_manager.cc:359] Metadata directory not provided
I20260811 19:00:44.124374 20154 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 19:00:44 dist-test-slave-h6n0 krb5kdc[18042](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474844, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.131@KRBTEST.COM for HTTP/127.29.112.148@KRBTEST.COM
I20260811 19:00:44.185534 20154 fs_manager.cc:709] Time spent opening directory manager: real 0.027s user 0.008s sys 0.016s
I20260811 19:00:44.199395 20184 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:44.204805 20154 fs_manager.cc:725] Time spent opening block manager: real 0.017s user 0.009s sys 0.003s
I20260811 19:00:44.205165 20154 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ts-2/wal
uuid: "2a3dc828d7cb4336841a377a442498b0"
format_stamp: "Formatted at 2026-08-11 19:00:40 on dist-test-slave-h6n0"
server_key: "402c4c24b31e9c1174610f90333e905f"
server_key_iv: "0a7a1624b0d2cfd94ba375ae045b171d"
server_key_version: "kuduclusterkey@0"
I20260811 19:00:44.206808 20154 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:00:44.267961 20154 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:00:44.274565 20154 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:00:44.275002 20154 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:00:44.287276 20154 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:00:44.289295 20154 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:00:44.289536 20154 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260811 19:00:44.289705 20154 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:00:44.289829 20154 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:44.347260 20154 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.131:38147
I20260811 19:00:44.347313 20374 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.131:38147 every 8 connection(s)
I20260811 19:00:44.348971 20154 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ts-2/data/info.pb
I20260811 19:00:44.356276 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 20154
Aug 11 19:00:44 dist-test-slave-h6n0 krb5kdc[18042](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474844, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.131@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 19:00:44.389729 20384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:44.358083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54992 (local address 127.29.112.190:36797)
0811 19:00:44.363195 (+ 5112us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:44.363201 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:44.363654 (+ 453us) server_negotiation.cc:407] Connection header received
0811 19:00:44.363945 (+ 291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:44.363950 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:44.364053 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:44.364160 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:44.369496 (+ 5336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:44.370775 (+ 1279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:44.372839 (+ 2064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:44.373476 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:44.373575 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:44.382815 (+ 9240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:44.382823 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:44.382828 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:44.382887 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:44.386386 (+ 3499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:44.387292 (+ 906us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:44.387298 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:44.387301 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:44.387447 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:44.387884 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:44.387889 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:44.387892 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:44.388264 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:44.388520 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:44.389106 (+ 586us) server_negotiation.cc:299] Negotiation successful
0811 19:00:44.389357 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4989,"thread_start_us":4854,"threads_started":1}
I20260811 19:00:44.390122 20381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:44.352552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.131:33389 (local address 127.29.112.190:36797)
0811 19:00:44.363005 (+ 10453us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:44.363013 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:44.364570 (+ 1557us) server_negotiation.cc:407] Connection header received
0811 19:00:44.367183 (+ 2613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:44.367190 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:44.367294 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:44.367431 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:44.369892 (+ 2461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:44.371349 (+ 1457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:44.374032 (+ 2683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:44.374812 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:44.374920 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:44.380163 (+ 5243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:44.380172 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:44.380177 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:44.380237 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:44.385452 (+ 5215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:44.386652 (+ 1200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:44.386659 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:44.386663 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:44.386831 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:44.387479 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:44.387485 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:44.387489 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:44.387887 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:44.388027 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:44.389215 (+ 1188us) server_negotiation.cc:299] Negotiation successful
0811 19:00:44.389542 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":45,"server-negotiator.queue_time_us":10155,"thread_start_us":9976,"threads_started":1}
I20260811 19:00:44.393852 20382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:44.353709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:36797 (local address 127.29.112.131:33389)
0811 19:00:44.364362 (+ 10653us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:44.364432 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:44.366839 (+ 2407us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:44.367973 (+ 1134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:44.367986 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:44.368716 (+ 730us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:44.369618 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:44.369646 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:44.371605 (+ 1959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:44.371610 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:44.373830 (+ 2220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:44.373848 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:44.375341 (+ 1493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:44.375350 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:44.375707 (+ 357us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:44.376521 (+ 814us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:44.376557 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:44.379833 (+ 3276us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:00:44.385686 (+ 5853us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:44.385698 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:44.385720 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:44.386457 (+ 737us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:44.386997 (+ 540us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:44.387002 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:44.387006 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:44.387307 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:44.388191 (+ 884us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:00:44.388201 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:00:44.388703 (+ 502us) client_negotiation.cc:769] Sending connection context
0811 19:00:44.390635 (+ 1932us) client_negotiation.cc:240] Negotiation successful
0811 19:00:44.391519 (+ 884us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9752,"thread_start_us":9584,"threads_started":1}
I20260811 19:00:44.395891 20378 heartbeater.cc:347] Connected to a master server at 127.29.112.190:36797
I20260811 19:00:44.396271 20378 heartbeater.cc:464] Registering TS with master...
I20260811 19:00:44.397307 20378 heartbeater.cc:511] Master 127.29.112.190:36797 requested a full tablet report, sending...
I20260811 19:00:44.399438 19598 ts_manager.cc:194] Registered new tserver with Master: 2a3dc828d7cb4336841a377a442498b0 (127.29.112.131:38147)
I20260811 19:00:44.400574 19598 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.131@KRBTEST.COM'} at 127.29.112.131:33389
I20260811 19:00:44.406929 30146 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 19:00:44.450827 20381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:44.428824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55002 (local address 127.29.112.190:36797)
0811 19:00:44.429149 (+ 325us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:44.429156 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:44.429553 (+ 397us) server_negotiation.cc:407] Connection header received
0811 19:00:44.429853 (+ 300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:44.429859 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:44.429959 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:44.430034 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:44.432551 (+ 2517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:44.433733 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:44.435850 (+ 2117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:44.436508 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:44.436620 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:44.440787 (+ 4167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:44.440795 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:44.440799 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:44.440854 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:44.445646 (+ 4792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:44.447146 (+ 1500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:44.447255 (+ 109us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:44.447264 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:44.447766 (+ 502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:44.448492 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:44.448502 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:44.448508 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:44.448990 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:44.449296 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:44.450078 (+ 782us) server_negotiation.cc:299] Negotiation successful
0811 19:00:44.450365 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169}
Aug 11 19:00:44 dist-test-slave-h6n0 krb5kdc[18042](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474843, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.190@KRBTEST.COM for HTTP/127.29.112.148@KRBTEST.COM
I20260811 19:00:44.458772 19598 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55002:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 19:00:44.475335 19598 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 19:00:44.574666 20151 heartbeater.cc:503] Master 127.29.112.190:36797 was elected leader, sending a full tablet report...
I20260811 19:00:44.590174 20401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:44.547865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.130:33745 (local address 127.0.0.1:52846)
0811 19:00:44.555260 (+ 7395us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:44.555315 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:44.555540 (+ 225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:44.567787 (+ 12247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:44.567797 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:44.567843 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:00:44.568379 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:44.568396 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:44.574676 (+ 6280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:44.574684 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:44.577493 (+ 2809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:44.577514 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:44.588975 (+ 11461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:44.588984 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:44.589275 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:44.589333 (+ 58us) client_negotiation.cc:769] Sending connection context
0811 19:00:44.589511 (+ 178us) client_negotiation.cc:240] Negotiation successful
0811 19:00:44.589746 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7049,"thread_start_us":6880,"threads_started":1}
I20260811 19:00:44.598193 20400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:44.547632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52846 (local address 127.29.112.130:33745)
0811 19:00:44.562287 (+ 14655us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:44.562299 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:44.562340 (+ 41us) server_negotiation.cc:407] Connection header received
0811 19:00:44.562471 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:44.562480 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:44.567016 (+ 4536us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:44.567371 (+ 355us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:00:44.569688 (+ 2317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:44.572600 (+ 2912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:44.577736 (+ 5136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:44.588215 (+ 10479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:44.588514 (+ 299us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:44.588723 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:44.597556 (+ 8833us) server_negotiation.cc:299] Negotiation successful
0811 19:00:44.597802 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14429,"thread_start_us":14912,"threads_started":1}
I20260811 19:00:44.602166 20401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:44.556208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.131:38147 (local address 127.0.0.1:35476)
0811 19:00:44.590954 (+ 34746us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:44.590982 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:44.591161 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:44.592503 (+ 1342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:44.592509 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:44.592542 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:00:44.592751 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:44.592763 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:44.594973 (+ 2210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:44.594979 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:44.597240 (+ 2261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:44.597264 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:44.601258 (+ 3994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:44.601266 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:44.601477 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:44.601519 (+ 42us) client_negotiation.cc:769] Sending connection context
0811 19:00:44.601628 (+ 109us) client_negotiation.cc:240] Negotiation successful
0811 19:00:44.601799 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34599,"threads_started":1}
I20260811 19:00:44.603518 20404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:44.556240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35476 (local address 127.29.112.131:38147)
0811 19:00:44.574277 (+ 18037us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:44.574288 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:44.591359 (+ 17071us) server_negotiation.cc:407] Connection header received
0811 19:00:44.591537 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:44.591550 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:44.591911 (+ 361us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:44.592140 (+ 229us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:00:44.592956 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:44.594787 (+ 1831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:44.598633 (+ 3846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:44.600360 (+ 1727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:44.600605 (+ 245us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:44.600803 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:44.602933 (+ 2130us) server_negotiation.cc:299] Negotiation successful
0811 19:00:44.603151 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17810,"thread_start_us":18056,"threads_started":1}
I20260811 19:00:44.616950 20271 tablet_service.cc:1467] Processing CreateTablet for tablet 7588991e22a54e429cd4341b02e144a1 (DEFAULT_TABLE table=test-table [id=f5ed4b4eda7846f8b3dc2e391df67c04]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:00:44.621850 20050 tablet_service.cc:1467] Processing CreateTablet for tablet 7588991e22a54e429cd4341b02e144a1 (DEFAULT_TABLE table=test-table [id=f5ed4b4eda7846f8b3dc2e391df67c04]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:00:44.624145 20050 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7588991e22a54e429cd4341b02e144a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:00:44.630731 20271 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7588991e22a54e429cd4341b02e144a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:00:44.650293 20401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:44.609853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:44275 (local address 127.0.0.1:54364)
0811 19:00:44.610153 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:44.610179 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:44.610337 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:44.642344 (+ 32007us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:44.642351 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:44.642390 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:00:44.642661 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:44.642677 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:44.645458 (+ 2781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:44.645465 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:44.647471 (+ 2006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:44.647491 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:44.649476 (+ 1985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:44.649485 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:44.649691 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:44.649729 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 19:00:44.649798 (+ 69us) client_negotiation.cc:240] Negotiation successful
0811 19:00:44.649969 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
I20260811 19:00:44.652441 20405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:44.603601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54364 (local address 127.29.112.129:44275)
0811 19:00:44.641531 (+ 37930us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:44.641542 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:44.641586 (+ 44us) server_negotiation.cc:407] Connection header received
0811 19:00:44.641708 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:44.641717 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:44.642052 (+ 335us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:44.642894 (+ 842us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:00:44.643150 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:44.645243 (+ 2093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:44.647722 (+ 2479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:44.649272 (+ 1550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:44.651053 (+ 1781us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:44.651294 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:44.651674 (+ 380us) server_negotiation.cc:299] Negotiation successful
0811 19:00:44.652017 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37706,"thread_start_us":37508,"threads_started":1}
I20260811 19:00:44.659150 19812 tablet_service.cc:1467] Processing CreateTablet for tablet 7588991e22a54e429cd4341b02e144a1 (DEFAULT_TABLE table=test-table [id=f5ed4b4eda7846f8b3dc2e391df67c04]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:00:44.663092 19812 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7588991e22a54e429cd4341b02e144a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:00:44.696296 20406 tablet_bootstrap.cc:492] T 7588991e22a54e429cd4341b02e144a1 P 5db07d6a145442db8d947d20919f8d75: Bootstrap starting.
I20260811 19:00:44.705406 20406 tablet_bootstrap.cc:654] T 7588991e22a54e429cd4341b02e144a1 P 5db07d6a145442db8d947d20919f8d75: Neither blocks nor log segments found. Creating new log.
I20260811 19:00:44.707351 20406 log.cc:824] T 7588991e22a54e429cd4341b02e144a1 P 5db07d6a145442db8d947d20919f8d75: Log is configured to *not* fsync() on all Append() calls
I20260811 19:00:44.710336 20407 tablet_bootstrap.cc:492] T 7588991e22a54e429cd4341b02e144a1 P 2a3dc828d7cb4336841a377a442498b0: Bootstrap starting.
I20260811 19:00:44.713690 20408 tablet_bootstrap.cc:492] T 7588991e22a54e429cd4341b02e144a1 P 9d2ed4daaa6b488b9bc634929daebf5c: Bootstrap starting.
I20260811 19:00:44.721550 20406 tablet_bootstrap.cc:492] T 7588991e22a54e429cd4341b02e144a1 P 5db07d6a145442db8d947d20919f8d75: No bootstrap required, opened a new log
I20260811 19:00:44.722245 20406 ts_tablet_manager.cc:1397] T 7588991e22a54e429cd4341b02e144a1 P 5db07d6a145442db8d947d20919f8d75: Time spent bootstrapping tablet: real 0.026s user 0.009s sys 0.009s
I20260811 19:00:44.735198 20408 tablet_bootstrap.cc:654] T 7588991e22a54e429cd4341b02e144a1 P 9d2ed4daaa6b488b9bc634929daebf5c: Neither blocks nor log segments found. Creating new log.
I20260811 19:00:44.736716 20407 tablet_bootstrap.cc:654] T 7588991e22a54e429cd4341b02e144a1 P 2a3dc828d7cb4336841a377a442498b0: Neither blocks nor log segments found. Creating new log.
I20260811 19:00:44.740288 20406 raft_consensus.cc:348] T 7588991e22a54e429cd4341b02e144a1 P 5db07d6a145442db8d947d20919f8d75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d2ed4daaa6b488b9bc634929daebf5c" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33745 } } peers { permanent_uuid: "2a3dc828d7cb4336841a377a442498b0" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 38147 } } peers { permanent_uuid: "5db07d6a145442db8d947d20919f8d75" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 44275 } }
I20260811 19:00:44.741072 20406 raft_consensus.cc:374] T 7588991e22a54e429cd4341b02e144a1 P 5db07d6a145442db8d947d20919f8d75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:00:44.741255 20406 raft_consensus.cc:729] T 7588991e22a54e429cd4341b02e144a1 P 5db07d6a145442db8d947d20919f8d75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5db07d6a145442db8d947d20919f8d75, State: Initialized, Role: FOLLOWER
I20260811 19:00:44.741304 20407 log.cc:824] T 7588991e22a54e429cd4341b02e144a1 P 2a3dc828d7cb4336841a377a442498b0: Log is configured to *not* fsync() on all Append() calls
I20260811 19:00:44.742240 20406 consensus_queue.cc:260] T 7588991e22a54e429cd4341b02e144a1 P 5db07d6a145442db8d947d20919f8d75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d2ed4daaa6b488b9bc634929daebf5c" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33745 } } peers { permanent_uuid: "2a3dc828d7cb4336841a377a442498b0" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 38147 } } peers { permanent_uuid: "5db07d6a145442db8d947d20919f8d75" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 44275 } }
I20260811 19:00:44.745093 20408 log.cc:824] T 7588991e22a54e429cd4341b02e144a1 P 9d2ed4daaa6b488b9bc634929daebf5c: Log is configured to *not* fsync() on all Append() calls
I20260811 19:00:44.751471 20407 tablet_bootstrap.cc:492] T 7588991e22a54e429cd4341b02e144a1 P 2a3dc828d7cb4336841a377a442498b0: No bootstrap required, opened a new log
I20260811 19:00:44.751612 20408 tablet_bootstrap.cc:492] T 7588991e22a54e429cd4341b02e144a1 P 9d2ed4daaa6b488b9bc634929daebf5c: No bootstrap required, opened a new log
I20260811 19:00:44.751925 20408 ts_tablet_manager.cc:1397] T 7588991e22a54e429cd4341b02e144a1 P 9d2ed4daaa6b488b9bc634929daebf5c: Time spent bootstrapping tablet: real 0.040s user 0.008s sys 0.003s
I20260811 19:00:44.751926 20407 ts_tablet_manager.cc:1397] T 7588991e22a54e429cd4341b02e144a1 P 2a3dc828d7cb4336841a377a442498b0: Time spent bootstrapping tablet: real 0.042s user 0.007s sys 0.004s
I20260811 19:00:44.759429 19929 heartbeater.cc:503] Master 127.29.112.190:36797 was elected leader, sending a full tablet report...
I20260811 19:00:44.761059 20406 ts_tablet_manager.cc:1428] T 7588991e22a54e429cd4341b02e144a1 P 5db07d6a145442db8d947d20919f8d75: Time spent starting tablet: real 0.039s user 0.023s sys 0.002s
I20260811 19:00:44.770294 20408 raft_consensus.cc:348] T 7588991e22a54e429cd4341b02e144a1 P 9d2ed4daaa6b488b9bc634929daebf5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d2ed4daaa6b488b9bc634929daebf5c" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33745 } } peers { permanent_uuid: "2a3dc828d7cb4336841a377a442498b0" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 38147 } } peers { permanent_uuid: "5db07d6a145442db8d947d20919f8d75" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 44275 } }
I20260811 19:00:44.771055 20408 raft_consensus.cc:374] T 7588991e22a54e429cd4341b02e144a1 P 9d2ed4daaa6b488b9bc634929daebf5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:00:44.771415 20408 raft_consensus.cc:729] T 7588991e22a54e429cd4341b02e144a1 P 9d2ed4daaa6b488b9bc634929daebf5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d2ed4daaa6b488b9bc634929daebf5c, State: Initialized, Role: FOLLOWER
I20260811 19:00:44.772373 20408 consensus_queue.cc:260] T 7588991e22a54e429cd4341b02e144a1 P 9d2ed4daaa6b488b9bc634929daebf5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d2ed4daaa6b488b9bc634929daebf5c" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33745 } } peers { permanent_uuid: "2a3dc828d7cb4336841a377a442498b0" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 38147 } } peers { permanent_uuid: "5db07d6a145442db8d947d20919f8d75" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 44275 } }
I20260811 19:00:44.779178 20407 raft_consensus.cc:348] T 7588991e22a54e429cd4341b02e144a1 P 2a3dc828d7cb4336841a377a442498b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d2ed4daaa6b488b9bc634929daebf5c" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33745 } } peers { permanent_uuid: "2a3dc828d7cb4336841a377a442498b0" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 38147 } } peers { permanent_uuid: "5db07d6a145442db8d947d20919f8d75" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 44275 } }
I20260811 19:00:44.780035 20407 raft_consensus.cc:374] T 7588991e22a54e429cd4341b02e144a1 P 2a3dc828d7cb4336841a377a442498b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:00:44.780222 20407 raft_consensus.cc:729] T 7588991e22a54e429cd4341b02e144a1 P 2a3dc828d7cb4336841a377a442498b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a3dc828d7cb4336841a377a442498b0, State: Initialized, Role: FOLLOWER
I20260811 19:00:44.781400 20407 consensus_queue.cc:260] T 7588991e22a54e429cd4341b02e144a1 P 2a3dc828d7cb4336841a377a442498b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d2ed4daaa6b488b9bc634929daebf5c" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33745 } } peers { permanent_uuid: "2a3dc828d7cb4336841a377a442498b0" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 38147 } } peers { permanent_uuid: "5db07d6a145442db8d947d20919f8d75" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 44275 } }
I20260811 19:00:44.784103 20408 ts_tablet_manager.cc:1428] T 7588991e22a54e429cd4341b02e144a1 P 9d2ed4daaa6b488b9bc634929daebf5c: Time spent starting tablet: real 0.032s user 0.026s sys 0.000s
I20260811 19:00:44.789098 20378 heartbeater.cc:503] Master 127.29.112.190:36797 was elected leader, sending a full tablet report...
I20260811 19:00:44.790119 20407 ts_tablet_manager.cc:1428] T 7588991e22a54e429cd4341b02e144a1 P 2a3dc828d7cb4336841a377a442498b0: Time spent starting tablet: real 0.038s user 0.013s sys 0.007s
I20260811 19:00:44.808179 20411 raft_consensus.cc:482] T 7588991e22a54e429cd4341b02e144a1 P 5db07d6a145442db8d947d20919f8d75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:00:44.808737 20411 raft_consensus.cc:504] T 7588991e22a54e429cd4341b02e144a1 P 5db07d6a145442db8d947d20919f8d75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d2ed4daaa6b488b9bc634929daebf5c" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33745 } } peers { permanent_uuid: "2a3dc828d7cb4336841a377a442498b0" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 38147 } } peers { permanent_uuid: "5db07d6a145442db8d947d20919f8d75" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 44275 } }
I20260811 19:00:44.810972 20411 leader_election.cc:288] T 7588991e22a54e429cd4341b02e144a1 P 5db07d6a145442db8d947d20919f8d75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9d2ed4daaa6b488b9bc634929daebf5c (127.29.112.130:33745), 2a3dc828d7cb4336841a377a442498b0 (127.29.112.131:38147)
I20260811 19:00:44.826584 20400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:44.812179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.129:56561 (local address 127.29.112.130:33745)
0811 19:00:44.812385 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:44.812391 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:44.816548 (+ 4157us) server_negotiation.cc:407] Connection header received
0811 19:00:44.816901 (+ 353us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:44.816908 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:44.817142 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:44.817231 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:00:44.819200 (+ 1969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:44.820292 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:44.823666 (+ 3374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:44.825068 (+ 1402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:44.825324 (+ 256us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:44.825423 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:44.826143 (+ 720us) server_negotiation.cc:299] Negotiation successful
0811 19:00:44.826287 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 19:00:44.827688 20422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:44.811556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.130:33745 (local address 127.29.112.129:56561)
0811 19:00:44.816394 (+ 4838us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:44.816422 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:44.816699 (+ 277us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:44.817481 (+ 782us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:44.817486 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:44.817516 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:00:44.818302 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:44.818319 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:44.821081 (+ 2762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:44.821088 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:44.823431 (+ 2343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:44.823451 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:44.825721 (+ 2270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:44.825728 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:44.825930 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:44.825964 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 19:00:44.827120 (+ 1156us) client_negotiation.cc:240] Negotiation successful
0811 19:00:44.827292 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4493,"thread_start_us":3950,"threads_started":1}
I20260811 19:00:44.835312 20085 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7588991e22a54e429cd4341b02e144a1" candidate_uuid: "5db07d6a145442db8d947d20919f8d75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d2ed4daaa6b488b9bc634929daebf5c" is_pre_election: true
I20260811 19:00:44.836004 20085 raft_consensus.cc:2454] T 7588991e22a54e429cd4341b02e144a1 P 9d2ed4daaa6b488b9bc634929daebf5c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5db07d6a145442db8d947d20919f8d75 in term 0.
I20260811 19:00:44.837105 19739 leader_election.cc:302] T 7588991e22a54e429cd4341b02e144a1 P 5db07d6a145442db8d947d20919f8d75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5db07d6a145442db8d947d20919f8d75, 9d2ed4daaa6b488b9bc634929daebf5c; no voters:
I20260811 19:00:44.837621 20411 raft_consensus.cc:2781] T 7588991e22a54e429cd4341b02e144a1 P 5db07d6a145442db8d947d20919f8d75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 19:00:44.837787 20411 raft_consensus.cc:482] T 7588991e22a54e429cd4341b02e144a1 P 5db07d6a145442db8d947d20919f8d75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 19:00:44.837881 20411 raft_consensus.cc:3037] T 7588991e22a54e429cd4341b02e144a1 P 5db07d6a145442db8d947d20919f8d75 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:00:44.841639 20411 raft_consensus.cc:504] T 7588991e22a54e429cd4341b02e144a1 P 5db07d6a145442db8d947d20919f8d75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d2ed4daaa6b488b9bc634929daebf5c" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33745 } } peers { permanent_uuid: "2a3dc828d7cb4336841a377a442498b0" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 38147 } } peers { permanent_uuid: "5db07d6a145442db8d947d20919f8d75" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 44275 } }
I20260811 19:00:44.843674 20085 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7588991e22a54e429cd4341b02e144a1" candidate_uuid: "5db07d6a145442db8d947d20919f8d75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d2ed4daaa6b488b9bc634929daebf5c"
I20260811 19:00:44.843925 20085 raft_consensus.cc:3037] T 7588991e22a54e429cd4341b02e144a1 P 9d2ed4daaa6b488b9bc634929daebf5c [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:00:44.848309 20085 raft_consensus.cc:2454] T 7588991e22a54e429cd4341b02e144a1 P 9d2ed4daaa6b488b9bc634929daebf5c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5db07d6a145442db8d947d20919f8d75 in term 1.
I20260811 19:00:44.848539 20411 leader_election.cc:288] T 7588991e22a54e429cd4341b02e144a1 P 5db07d6a145442db8d947d20919f8d75 [CANDIDATE]: Term 1 election: Requested vote from peers 9d2ed4daaa6b488b9bc634929daebf5c (127.29.112.130:33745), 2a3dc828d7cb4336841a377a442498b0 (127.29.112.131:38147)
I20260811 19:00:44.849249 19739 leader_election.cc:302] T 7588991e22a54e429cd4341b02e144a1 P 5db07d6a145442db8d947d20919f8d75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5db07d6a145442db8d947d20919f8d75, 9d2ed4daaa6b488b9bc634929daebf5c; no voters:
I20260811 19:00:44.849715 20411 raft_consensus.cc:2781] T 7588991e22a54e429cd4341b02e144a1 P 5db07d6a145442db8d947d20919f8d75 [term 1 FOLLOWER]: Leader election won for term 1
W20260811 19:00:44.853142 20379 tablet.cc:2369] T 7588991e22a54e429cd4341b02e144a1 P 2a3dc828d7cb4336841a377a442498b0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 19:00:44.863600 20411 raft_consensus.cc:686] T 7588991e22a54e429cd4341b02e144a1 P 5db07d6a145442db8d947d20919f8d75 [term 1 LEADER]: Becoming Leader. State: Replica: 5db07d6a145442db8d947d20919f8d75, State: Running, Role: LEADER
I20260811 19:00:44.864320 20404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:44.816709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.129:44599 (local address 127.29.112.131:38147)
0811 19:00:44.816966 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:44.816977 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:44.829181 (+ 12204us) server_negotiation.cc:407] Connection header received
0811 19:00:44.829459 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:44.829465 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:44.829547 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:44.829612 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:00:44.850663 (+ 21051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:44.851792 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:44.861201 (+ 9409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:44.862602 (+ 1401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:44.862761 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:44.862848 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:44.863689 (+ 841us) server_negotiation.cc:299] Negotiation successful
0811 19:00:44.863850 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 19:00:44.864657 20411 consensus_queue.cc:237] T 7588991e22a54e429cd4341b02e144a1 P 5db07d6a145442db8d947d20919f8d75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d2ed4daaa6b488b9bc634929daebf5c" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33745 } } peers { permanent_uuid: "2a3dc828d7cb4336841a377a442498b0" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 38147 } } peers { permanent_uuid: "5db07d6a145442db8d947d20919f8d75" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 44275 } }
I20260811 19:00:44.865609 20422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:44.816951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.131:38147 (local address 127.29.112.129:44599)
0811 19:00:44.829056 (+ 12105us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:44.829079 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:44.829300 (+ 221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:44.833721 (+ 4421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:44.833726 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:44.833766 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:00:44.834135 (+ 369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:44.834147 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:44.852516 (+ 18369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:44.852527 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:44.860259 (+ 7732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:44.860291 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:44.863037 (+ 2746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:44.863045 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:44.863424 (+ 379us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:44.863468 (+ 44us) client_negotiation.cc:769] Sending connection context
0811 19:00:44.864912 (+ 1444us) client_negotiation.cc:240] Negotiation successful
0811 19:00:44.865159 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11978,"thread_start_us":16694,"threads_started":1}
I20260811 19:00:44.867336 20301 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7588991e22a54e429cd4341b02e144a1" candidate_uuid: "5db07d6a145442db8d947d20919f8d75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a3dc828d7cb4336841a377a442498b0"
I20260811 19:00:44.867573 20305 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7588991e22a54e429cd4341b02e144a1" candidate_uuid: "5db07d6a145442db8d947d20919f8d75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a3dc828d7cb4336841a377a442498b0" is_pre_election: true
I20260811 19:00:44.867914 20301 raft_consensus.cc:3037] T 7588991e22a54e429cd4341b02e144a1 P 2a3dc828d7cb4336841a377a442498b0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:00:44.871431 20301 raft_consensus.cc:2454] T 7588991e22a54e429cd4341b02e144a1 P 2a3dc828d7cb4336841a377a442498b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5db07d6a145442db8d947d20919f8d75 in term 1.
I20260811 19:00:44.876371 19595 catalog_manager.cc:5495] T 7588991e22a54e429cd4341b02e144a1 P 5db07d6a145442db8d947d20919f8d75 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5db07d6a145442db8d947d20919f8d75 (127.29.112.129). New cstate: current_term: 1 leader_uuid: "5db07d6a145442db8d947d20919f8d75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d2ed4daaa6b488b9bc634929daebf5c" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2a3dc828d7cb4336841a377a442498b0" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 38147 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5db07d6a145442db8d947d20919f8d75" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 44275 } health_report { overall_health: HEALTHY } } }
I20260811 19:00:44.950237 20405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:44.941452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54372 (local address 127.29.112.129:44275)
0811 19:00:44.941777 (+ 325us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:44.941785 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:44.941811 (+ 26us) server_negotiation.cc:407] Connection header received
0811 19:00:44.942059 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:44.942066 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:44.942196 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:44.942382 (+ 186us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:00:44.943096 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:44.944303 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:44.946661 (+ 2358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:44.947334 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:44.947477 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:44.948446 (+ 969us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:00:44.949171 (+ 725us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:00:44.949314 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:44.949837 (+ 523us) server_negotiation.cc:299] Negotiation successful
0811 19:00:44.949930 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153}
W20260811 19:00:44.962909 19930 tablet.cc:2369] T 7588991e22a54e429cd4341b02e144a1 P 5db07d6a145442db8d947d20919f8d75: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 19:00:44.975636 20085 raft_consensus.cc:1261] T 7588991e22a54e429cd4341b02e144a1 P 9d2ed4daaa6b488b9bc634929daebf5c [term 1 FOLLOWER]: Refusing update from remote peer 5db07d6a145442db8d947d20919f8d75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:00:44.977044 20425 consensus_queue.cc:1035] T 7588991e22a54e429cd4341b02e144a1 P 5db07d6a145442db8d947d20919f8d75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d2ed4daaa6b488b9bc634929daebf5c" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:00:44.978698 20301 raft_consensus.cc:1261] T 7588991e22a54e429cd4341b02e144a1 P 2a3dc828d7cb4336841a377a442498b0 [term 1 FOLLOWER]: Refusing update from remote peer 5db07d6a145442db8d947d20919f8d75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:00:44.979705 20425 consensus_queue.cc:1035] T 7588991e22a54e429cd4341b02e144a1 P 5db07d6a145442db8d947d20919f8d75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a3dc828d7cb4336841a377a442498b0" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 38147 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 19:00:45.018734 20156 tablet.cc:2369] T 7588991e22a54e429cd4341b02e144a1 P 9d2ed4daaa6b488b9bc634929daebf5c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 19:00:45.036051 20429 mvcc.cc:205] Tried to move back new op lower bound from 7317400964994699264 to 7317400964572372992. Current Snapshot: MvccSnapshot[applied={T|T < 7317400964994699264}]
I20260811 19:00:45.043984 20431 mvcc.cc:205] Tried to move back new op lower bound from 7317400964994699264 to 7317400964572372992. Current Snapshot: MvccSnapshot[applied={T|T < 7317400964994699264}]
I20260811 19:00:45.072477 20432 mvcc.cc:205] Tried to move back new op lower bound from 7317400964994699264 to 7317400964572372992. Current Snapshot: MvccSnapshot[applied={T|T < 7317400964994699264}]
I20260811 19:00:45.089730 19595 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55002:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:00:45.090299 19595 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55002:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:00:45.101706 19595 catalog_manager.cc:5782] T 00000000000000000000000000000000 P daba4ff62b2d4177855a7a8bc0ed5bde: Sending DeleteTablet for 3 replicas of tablet 7588991e22a54e429cd4341b02e144a1
I20260811 19:00:45.103550 20050 tablet_service.cc:1514] Processing DeleteTablet for tablet 7588991e22a54e429cd4341b02e144a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:00:45 UTC) from {username='kudu', principal='kudu/127.29.112.190@KRBTEST.COM'} at 127.0.0.1:52846
I20260811 19:00:45.104453 20271 tablet_service.cc:1514] Processing DeleteTablet for tablet 7588991e22a54e429cd4341b02e144a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:00:45 UTC) from {username='kudu', principal='kudu/127.29.112.190@KRBTEST.COM'} at 127.0.0.1:35476
I20260811 19:00:45.104887 19812 tablet_service.cc:1514] Processing DeleteTablet for tablet 7588991e22a54e429cd4341b02e144a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:00:45 UTC) from {username='kudu', principal='kudu/127.29.112.190@KRBTEST.COM'} at 127.0.0.1:54364
I20260811 19:00:45.107575 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 19656
I20260811 19:00:45.146026 20456 tablet_replica.cc:330] T 7588991e22a54e429cd4341b02e144a1 P 2a3dc828d7cb4336841a377a442498b0: stopping tablet replica
I20260811 19:00:45.147141 20454 tablet_replica.cc:330] T 7588991e22a54e429cd4341b02e144a1 P 9d2ed4daaa6b488b9bc634929daebf5c: stopping tablet replica
I20260811 19:00:45.148020 20456 raft_consensus.cc:2229] T 7588991e22a54e429cd4341b02e144a1 P 2a3dc828d7cb4336841a377a442498b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:00:45.148777 20454 raft_consensus.cc:2229] T 7588991e22a54e429cd4341b02e144a1 P 9d2ed4daaa6b488b9bc634929daebf5c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:00:45.149626 20456 raft_consensus.cc:2258] T 7588991e22a54e429cd4341b02e144a1 P 2a3dc828d7cb4336841a377a442498b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:00:45.150291 20454 raft_consensus.cc:2258] T 7588991e22a54e429cd4341b02e144a1 P 9d2ed4daaa6b488b9bc634929daebf5c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:00:45.151081 20457 tablet_replica.cc:330] T 7588991e22a54e429cd4341b02e144a1 P 5db07d6a145442db8d947d20919f8d75: stopping tablet replica
I20260811 19:00:45.151503 20457 raft_consensus.cc:2229] T 7588991e22a54e429cd4341b02e144a1 P 5db07d6a145442db8d947d20919f8d75 [term 1 LEADER]: Raft consensus shutting down.
I20260811 19:00:45.152133 20457 raft_consensus.cc:2258] T 7588991e22a54e429cd4341b02e144a1 P 5db07d6a145442db8d947d20919f8d75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:00:45.155951 20454 ts_tablet_manager.cc:1905] T 7588991e22a54e429cd4341b02e144a1 P 9d2ed4daaa6b488b9bc634929daebf5c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:00:45.163727 20456 ts_tablet_manager.cc:1905] T 7588991e22a54e429cd4341b02e144a1 P 2a3dc828d7cb4336841a377a442498b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:00:45.167994 20454 ts_tablet_manager.cc:1918] T 7588991e22a54e429cd4341b02e144a1 P 9d2ed4daaa6b488b9bc634929daebf5c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:00:45.168550 20454 log.cc:1196] T 7588991e22a54e429cd4341b02e144a1 P 9d2ed4daaa6b488b9bc634929daebf5c: Deleting WAL directory at /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ts-1/wal/wals/7588991e22a54e429cd4341b02e144a1
I20260811 19:00:45.169299 20454 ts_tablet_manager.cc:1939] T 7588991e22a54e429cd4341b02e144a1 P 9d2ed4daaa6b488b9bc634929daebf5c: Deleting consensus metadata
I20260811 19:00:45.170847 20457 ts_tablet_manager.cc:1905] T 7588991e22a54e429cd4341b02e144a1 P 5db07d6a145442db8d947d20919f8d75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:00:45.171718 19568 catalog_manager.cc:4841] TS 9d2ed4daaa6b488b9bc634929daebf5c (127.29.112.130:33745): tablet 7588991e22a54e429cd4341b02e144a1 (table test-table [id=f5ed4b4eda7846f8b3dc2e391df67c04]) successfully deleted
I20260811 19:00:45.185478 20457 ts_tablet_manager.cc:1918] T 7588991e22a54e429cd4341b02e144a1 P 5db07d6a145442db8d947d20919f8d75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:00:45.186641 20456 ts_tablet_manager.cc:1918] T 7588991e22a54e429cd4341b02e144a1 P 2a3dc828d7cb4336841a377a442498b0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:00:45.185874 20405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:45.109548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54374 (local address 127.29.112.129:44275)
0811 19:00:45.109809 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:45.109820 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:45.140183 (+ 30363us) server_negotiation.cc:407] Connection header received
0811 19:00:45.140611 (+ 428us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:45.140621 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:45.140784 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:45.140908 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:45.154975 (+ 14067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:45.156407 (+ 1432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:45.172099 (+ 15692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:45.172705 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:45.172815 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:45.176968 (+ 4153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:45.176988 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:45.177010 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:45.177084 (+ 74us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:45.181532 (+ 4448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:45.182725 (+ 1193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:45.182738 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:45.182744 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:45.183015 (+ 271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:45.183614 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:45.183619 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:45.183622 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:45.184138 (+ 516us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:45.184476 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:45.185124 (+ 648us) server_negotiation.cc:299] Negotiation successful
0811 19:00:45.185394 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260811 19:00:45.187183 20456 log.cc:1196] T 7588991e22a54e429cd4341b02e144a1 P 2a3dc828d7cb4336841a377a442498b0: Deleting WAL directory at /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ts-2/wal/wals/7588991e22a54e429cd4341b02e144a1
I20260811 19:00:45.187676 19921 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:00:45.188027 20456 ts_tablet_manager.cc:1939] T 7588991e22a54e429cd4341b02e144a1 P 2a3dc828d7cb4336841a377a442498b0: Deleting consensus metadata
I20260811 19:00:45.190165 19568 catalog_manager.cc:4841] TS 2a3dc828d7cb4336841a377a442498b0 (127.29.112.131:38147): tablet 7588991e22a54e429cd4341b02e144a1 (table test-table [id=f5ed4b4eda7846f8b3dc2e391df67c04]) successfully deleted
I20260811 19:00:45.190797 20457 log.cc:1196] T 7588991e22a54e429cd4341b02e144a1 P 5db07d6a145442db8d947d20919f8d75: Deleting WAL directory at /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474615205622-30146-0/minicluster-data/ts-0/wal/wals/7588991e22a54e429cd4341b02e144a1
I20260811 19:00:45.316815 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 19656
I20260811 19:00:45.363554 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 19933
W20260811 19:00:45.363525 19568 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.112.129:44275 (error 108)
I20260811 19:00:45.390349 20462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:45.384808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:44275 (local address 127.0.0.1:54378)
0811 19:00:45.389899 (+ 5091us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:45.390079 (+ 180us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.129:44275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4935,"threads_started":1}
W20260811 19:00:45.390966 19568 catalog_manager.cc:4584] TS 5db07d6a145442db8d947d20919f8d75 (127.29.112.129:44275): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7588991e22a54e429cd4341b02e144a1: Network error: Client connection negotiation failed: client connection to 127.29.112.129:44275: connect: Connection refused (error 111)
I20260811 19:00:45.397358 20461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:45.364726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52854 (local address 127.29.112.130:33745)
0811 19:00:45.376332 (+ 11606us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:45.376339 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:45.376371 (+ 32us) server_negotiation.cc:407] Connection header received
0811 19:00:45.376452 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:45.376457 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:45.376563 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:45.378903 (+ 2340us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:45.379076 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:45.380123 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:45.382089 (+ 1966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:45.382919 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:45.387308 (+ 4389us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:45.387593 (+ 285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:45.387605 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:45.387624 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:45.387660 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:45.392473 (+ 4813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:45.393676 (+ 1203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:45.393687 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:45.393691 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:45.393901 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:45.394692 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:45.394698 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:45.394702 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:45.395364 (+ 662us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:45.395731 (+ 367us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:45.396384 (+ 653us) server_negotiation.cc:299] Negotiation successful
0811 19:00:45.396875 (+ 491us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11442,"thread_start_us":12051,"threads_started":1}
I20260811 19:00:45.398380 20142 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:00:45.444967 20462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:45.444330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:44275 (local address 127.0.0.1:54384)
0811 19:00:45.444619 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:45.444762 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.129:44275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122}
I20260811 19:00:45.446306 20462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:45.445849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:44275 (local address 127.0.0.1:54400)
0811 19:00:45.446030 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:45.446134 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.129:44275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260811 19:00:45.507534 20462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:45.506824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:44275 (local address 127.0.0.1:54412)
0811 19:00:45.507144 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:45.507261 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.129:44275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125}
I20260811 19:00:45.509708 20462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:45.509100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:44275 (local address 127.0.0.1:54414)
0811 19:00:45.509404 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:45.509518 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.129:44275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260811 19:00:45.512100 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 19933
I20260811 19:00:45.551736 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 20154
I20260811 19:00:45.573326 20465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:45.552874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35492 (local address 127.29.112.131:38147)
0811 19:00:45.553561 (+ 687us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:45.553567 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:45.553585 (+ 18us) server_negotiation.cc:407] Connection header received
0811 19:00:45.553645 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:45.553649 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:45.553717 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:45.553864 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:45.556753 (+ 2889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:45.557625 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:45.560345 (+ 2720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:45.560862 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:45.561294 (+ 432us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:45.565094 (+ 3800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:45.565115 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:45.565134 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:45.565180 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:45.569370 (+ 4190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:45.570315 (+ 945us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:45.570324 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:45.570326 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:45.570492 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:45.571257 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:45.571266 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:45.571270 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:45.571718 (+ 448us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:45.572123 (+ 405us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:45.572473 (+ 350us) server_negotiation.cc:299] Negotiation successful
0811 19:00:45.572740 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":566,"thread_start_us":451,"threads_started":1}
I20260811 19:00:45.574390 20365 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:00:45.592556 20462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:45.591771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:44275 (local address 127.0.0.1:54416)
0811 19:00:45.592057 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:45.592159 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.129:44275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132}
I20260811 19:00:45.595013 20462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:45.594113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:44275 (local address 127.0.0.1:54430)
0811 19:00:45.594703 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:45.594809 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.129:44275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355}
I20260811 19:00:45.691953 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 20154
I20260811 19:00:45.726594 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 19540
I20260811 19:00:45.728000 19643 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:00:45.859961 20462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:45.858994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:44275 (local address 127.0.0.1:54440)
0811 19:00:45.859634 (+ 640us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:45.859781 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.129:44275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":295}
I20260811 19:00:45.860193 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 19540
I20260811 19:00:45.902388 30146 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260811 19:00:46.262488 30146 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260811 19:00:46.262910 30146 mini_ranger.cc:66] Stopping Ranger...
I20260811 19:00:46.623090 30146 mini_ranger.cc:68] Stopped Ranger
2026-08-11 19:00:46.623 UTC [18093] LOG: received smart shutdown request
2026-08-11 19:00:46.630 UTC [18093] LOG: background worker "logical replication launcher" (PID 18107) exited with exit code 1
2026-08-11 19:00:46.631 UTC [18098] LOG: shutting down
2026-08-11 19:00:46.987 UTC [18093] LOG: database system is shut down
2026-08-11T19:00:46Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (66236 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474615205622-30146-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:00:47 dist-test-slave-h6n0 krb5kdc[20482](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:00:47 dist-test-slave-h6n0 krb5kdc[20482](info): set up 2 sockets
Aug 11 19:00:47 dist-test-slave-h6n0 krb5kdc[20482](info): commencing operation
krb5kdc: starting...
W20260811 19:00:48.762699 30146 mini_kdc.cc:121] Time spent starting KDC: real 1.548s user 0.003s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474615205622-30146-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474615205622-30146-0/krb5kdc/test-user.keytab.
Aug 11 19:00:48 dist-test-slave-h6n0 krb5kdc[20482](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474848, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:00:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:00:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.112.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab.
Entry for principal kudu/127.29.112.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab.
I20260811 19:00:49.147272 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474615205622-30146-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474615205622-30146-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474615205622-30146-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.190:45847
--webserver_interface=127.29.112.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab
--principal=kudu/127.29.112.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:40535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.190:45847,127.29.112.189:33049,127.29.112.188:37973
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:00:49.616536 20497 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:00:49.616937 20497 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:00:49.617138 20497 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:00:49.627969 20497 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:00:49.628190 20497 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:00:49.628321 20497 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:00:49.628451 20497 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:00:49.628504 20497 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:00:49.644824 20497 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:40535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474615205622-30146-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474615205622-30146-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.112.190:45847,127.29.112.189:33049,127.29.112.188:37973
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab
--principal=kudu/127.29.112.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.190:45847
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.112.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474615205622-30146-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:00:49.647774 20497 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:00:49.650204 20497 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:00:49.669274 20506 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:49.669960 20508 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:49.670230 20505 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:49.673444 20497 server_base.cc:1034] running on GCE node
I20260811 19:00:49.675246 20497 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:00:49.677146 20497 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:00:49.678664 20497 hybrid_clock.cc:648] HybridClock initialized: now 1786474849678498 us; error 97 us; skew 500 ppm
I20260811 19:00:49.679169 20497 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:00:49 dist-test-slave-h6n0 krb5kdc[20482](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474849, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:00:49.689028 20497 init.cc:377] Logged in from keytab as kudu/127.29.112.190@KRBTEST.COM (short username kudu)
I20260811 19:00:49.691797 20497 webserver.cc:460] Webserver started at http://127.29.112.190:40791/ using document root <none> and password file <none>
I20260811 19:00:49.692786 20497 fs_manager.cc:359] Metadata directory not provided
I20260811 19:00:49.693010 20497 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:00:49.693436 20497 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:00:49.697261 20497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474615205622-30146-0/minicluster-data/master-0/data/instance:
uuid: "312aa2c1b36b44998c5e3e4f77f36d88"
format_stamp: "Formatted at 2026-08-11 19:00:49 on dist-test-slave-h6n0"
server_key: "fb4b31b888d58912a5af9a00e7f0f999"
server_key_iv: "8fe3cd2a25e3eaf417d7450fcba981ca"
server_key_version: "encryptionkey@0"
I20260811 19:00:49.698316 20497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474615205622-30146-0/minicluster-data/master-0/wal/instance:
uuid: "312aa2c1b36b44998c5e3e4f77f36d88"
format_stamp: "Formatted at 2026-08-11 19:00:49 on dist-test-slave-h6n0"
server_key: "fb4b31b888d58912a5af9a00e7f0f999"
server_key_iv: "8fe3cd2a25e3eaf417d7450fcba981ca"
server_key_version: "encryptionkey@0"
I20260811 19:00:49.707281 20497 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.007s sys 0.000s
I20260811 19:00:49.712893 20523 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:49.714818 20497 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 19:00:49.715086 20497 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474615205622-30146-0/minicluster-data/master-0/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474615205622-30146-0/minicluster-data/master-0/wal
uuid: "312aa2c1b36b44998c5e3e4f77f36d88"
format_stamp: "Formatted at 2026-08-11 19:00:49 on dist-test-slave-h6n0"
server_key: "fb4b31b888d58912a5af9a00e7f0f999"
server_key_iv: "8fe3cd2a25e3eaf417d7450fcba981ca"
server_key_version: "encryptionkey@0"
I20260811 19:00:49.715360 20497 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474615205622-30146-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474615205622-30146-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474615205622-30146-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:00:49.751734 20497 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:00:49.757819 20497 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:00:49.758230 20497 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:00:49.788971 20497 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.190:45847
I20260811 19:00:49.789001 20609 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.190:45847 every 8 connection(s)
I20260811 19:00:49.791712 20497 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
I20260811 19:00:49.797886 20610 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:00:49.802376 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 20497
I20260811 19:00:49.802773 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474615205622-30146-0/minicluster-data/master-0/wal/instance
I20260811 19:00:49.804311 30146 external_mini_cluster.cc:1428] Setting key d1611b92a2ffa3388f85b02acddad3b3
I20260811 19:00:49.811695 20610 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.112.190" port: 45847 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 19:00:49 dist-test-slave-h6n0 krb5kdc[20482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474849, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.190@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.29.112.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.189@KRBTEST.COM" created.
I20260811 19:00:49.850309 20615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:49.816557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:45847 (local address 127.0.0.1:44336)
0811 19:00:49.819689 (+ 3132us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:49.819791 (+ 102us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:49.822867 (+ 3076us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:49.824781 (+ 1914us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:49.824792 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:49.826070 (+ 1278us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:49.826760 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:49.826799 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:49.829389 (+ 2590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:49.829397 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:49.831926 (+ 2529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:49.831952 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:49.833714 (+ 1762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:49.833731 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:49.834061 (+ 330us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:49.835199 (+ 1138us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:49.835253 (+ 54us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:49.839389 (+ 4136us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:00:49.844474 (+ 5085us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:49.844491 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:49.844535 (+ 44us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:49.845068 (+ 533us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:49.845581 (+ 513us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:49.845585 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:49.845590 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:49.845744 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:49.846816 (+ 1072us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:00:49.846830 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:00:49.847124 (+ 294us) client_negotiation.cc:769] Sending connection context
0811 19:00:49.847402 (+ 278us) client_negotiation.cc:240] Negotiation successful
0811 19:00:49.848233 (+ 831us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2722,"mutex_wait_us":177,"thread_start_us":2372,"threads_started":1}
I20260811 19:00:49.850203 20617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:49.816547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44336 (local address 127.29.112.190:45847)
0811 19:00:49.821198 (+ 4651us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:49.821214 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:49.821290 (+ 76us) server_negotiation.cc:407] Connection header received
0811 19:00:49.823874 (+ 2584us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:49.823884 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:49.824447 (+ 563us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:49.824664 (+ 217us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:49.827141 (+ 2477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:49.829081 (+ 1940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:49.832252 (+ 3171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:49.833133 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:49.833317 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:49.839703 (+ 6386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:49.839712 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:49.839728 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:49.839763 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:49.843992 (+ 4229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:49.845241 (+ 1249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:49.845258 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:49.845260 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:49.845429 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:49.845870 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:49.845876 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:49.845881 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:49.846654 (+ 773us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:49.846916 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:49.847541 (+ 625us) server_negotiation.cc:299] Negotiation successful
0811 19:00:49.847928 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4359,"thread_start_us":4076,"threads_started":1}
I20260811 19:00:49.855170 20610 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.112.189" port: 33049 } has no permanent_uuid. Determining permanent_uuid...
I20260811 19:00:49.856931 20615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:49.856314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.189:33049 (local address 127.0.0.1:46982)
0811 19:00:49.856575 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:49.856743 (+ 168us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.189:33049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
W20260811 19:00:49.857437 20526 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.112.189:33049: connect: Connection refused (error 111)
I20260811 19:00:49.859830 20615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:49.859287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.189:33049 (local address 127.0.0.1:46998)
0811 19:00:49.859526 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:49.859627 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.189:33049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
W20260811 19:00:49.860121 20610 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.112.189:33049: Network error: Client connection negotiation failed: client connection to 127.29.112.189:33049: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.189.keytab.
Entry for principal kudu/127.29.112.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.189.keytab.
I20260811 19:00:49.896052 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474615205622-30146-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474615205622-30146-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474615205622-30146-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474615205622-30146-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.189:33049
--webserver_interface=127.29.112.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.189.keytab
--principal=kudu/127.29.112.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:40535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.190:45847,127.29.112.189:33049,127.29.112.188:37973
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 19:00:49.909575 20610 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.112.189" port: 33049 } attempt: 1
I20260811 19:00:49.911005 20615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:49.910449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.189:33049 (local address 127.0.0.1:47002)
0811 19:00:49.910676 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:49.910851 (+ 175us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.189:33049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260811 19:00:49.912421 20615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:49.911952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.189:33049 (local address 127.0.0.1:47014)
0811 19:00:49.912135 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:49.912234 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.189:33049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260811 19:00:49.912657 20610 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.112.189:33049: Network error: Client connection negotiation failed: client connection to 127.29.112.189:33049: connect: Connection refused (error 111)
I20260811 19:00:49.981395 20610 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.112.189" port: 33049 } attempt: 2
I20260811 19:00:49.983035 20615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:49.982275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.189:33049 (local address 127.0.0.1:47030)
0811 19:00:49.982710 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:49.982841 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.189:33049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260811 19:00:49.984956 20615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:49.983897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.189:33049 (local address 127.0.0.1:47032)
0811 19:00:49.984455 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:49.984634 (+ 179us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.189:33049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
W20260811 19:00:49.985455 20610 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.112.189:33049: Network error: Client connection negotiation failed: client connection to 127.29.112.189:33049: connect: Connection refused (error 111)
I20260811 19:00:50.076970 20610 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.112.189" port: 33049 } attempt: 3
I20260811 19:00:50.078748 20615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:50.077932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.189:33049 (local address 127.0.0.1:47048)
0811 19:00:50.078221 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:50.078347 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.189:33049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147}
I20260811 19:00:50.080085 20615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:50.079631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.189:33049 (local address 127.0.0.1:47064)
0811 19:00:50.079831 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:50.079920 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.189:33049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
W20260811 19:00:50.080387 20610 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.112.189:33049: Network error: Client connection negotiation failed: client connection to 127.29.112.189:33049: connect: Connection refused (error 111)
I20260811 19:00:50.223762 20610 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.112.189" port: 33049 } attempt: 4
I20260811 19:00:50.225711 20615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:50.224916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.189:33049 (local address 127.0.0.1:47074)
0811 19:00:50.225383 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:50.225520 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.189:33049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":254}
I20260811 19:00:50.227596 20615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:50.226880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.189:33049 (local address 127.0.0.1:47088)
0811 19:00:50.227060 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:50.227401 (+ 341us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.189:33049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260811 19:00:50.228008 20610 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.112.189:33049: Network error: Client connection negotiation failed: client connection to 127.29.112.189:33049: connect: Connection refused (error 111)
W20260811 19:00:50.376133 20624 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:00:50.376516 20624 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:00:50.376739 20624 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:00:50.387426 20624 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:00:50.387594 20624 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:00:50.387647 20624 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:00:50.387794 20624 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:00:50.387888 20624 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:00:50.408411 20624 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:40535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474615205622-30146-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474615205622-30146-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.29.112.190:45847,127.29.112.189:33049,127.29.112.188:37973
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.189.keytab
--principal=kudu/127.29.112.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.189:33049
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474615205622-30146-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.29.112.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474615205622-30146-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:00:50.410004 20624 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:00:50.411540 20624 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:00:50.429553 20630 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:50.429733 20634 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:50.430027 20631 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:50.431643 20624 server_base.cc:1034] running on GCE node
I20260811 19:00:50.432798 20624 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:00:50.434707 20624 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:00:50.436040 20624 hybrid_clock.cc:648] HybridClock initialized: now 1786474850435931 us; error 99 us; skew 500 ppm
I20260811 19:00:50.436520 20624 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:00:50 dist-test-slave-h6n0 krb5kdc[20482](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474850, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:00:50.441314 20624 init.cc:377] Logged in from keytab as kudu/127.29.112.189@KRBTEST.COM (short username kudu)
I20260811 19:00:50.443783 20624 webserver.cc:460] Webserver started at http://127.29.112.189:34329/ using document root <none> and password file <none>
I20260811 19:00:50.444664 20624 fs_manager.cc:359] Metadata directory not provided
I20260811 19:00:50.444840 20624 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:00:50.445220 20624 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:00:50.449431 20624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474615205622-30146-0/minicluster-data/master-1/data/instance:
uuid: "6969cb95acbc49fa98783fb8853482c9"
format_stamp: "Formatted at 2026-08-11 19:00:50 on dist-test-slave-h6n0"
server_key: "cfa78838c84991a2f13a778902e17e1c"
server_key_iv: "8c9a6660c854336663fada700a1dc3a2"
server_key_version: "encryptionkey@0"
I20260811 19:00:50.450414 20624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474615205622-30146-0/minicluster-data/master-1/wal/instance:
uuid: "6969cb95acbc49fa98783fb8853482c9"
format_stamp: "Formatted at 2026-08-11 19:00:50 on dist-test-slave-h6n0"
server_key: "cfa78838c84991a2f13a778902e17e1c"
server_key_iv: "8c9a6660c854336663fada700a1dc3a2"
server_key_version: "encryptionkey@0"
I20260811 19:00:50.457077 20624 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 19:00:50.462703 20646 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:50.465498 20624 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.001s
I20260811 19:00:50.465787 20624 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474615205622-30146-0/minicluster-data/master-1/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474615205622-30146-0/minicluster-data/master-1/wal
uuid: "6969cb95acbc49fa98783fb8853482c9"
format_stamp: "Formatted at 2026-08-11 19:00:50 on dist-test-slave-h6n0"
server_key: "cfa78838c84991a2f13a778902e17e1c"
server_key_iv: "8c9a6660c854336663fada700a1dc3a2"
server_key_version: "encryptionkey@0"
I20260811 19:00:50.466038 20624 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474615205622-30146-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474615205622-30146-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474615205622-30146-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:00:50.482983 20624 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:00:50.489919 20624 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:00:50.490221 20624 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:00:50.519971 20624 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.189:33049
I20260811 19:00:50.520310 20746 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.189:33049 every 8 connection(s)
I20260811 19:00:50.523010 20624 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474615205622-30146-0/minicluster-data/master-1/data/info.pb
I20260811 19:00:50.527484 20610 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.112.189" port: 33049 } attempt: 5
I20260811 19:00:50.528939 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 20624
I20260811 19:00:50.529186 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474615205622-30146-0/minicluster-data/master-1/wal/instance
I20260811 19:00:50.529860 30146 external_mini_cluster.cc:1428] Setting key e58da212e263bb88db105da328cb5436
I20260811 19:00:50.530383 20750 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:00:50.543960 20750 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.112.190" port: 45847 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 19:00:50 dist-test-slave-h6n0 krb5kdc[20482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474849, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.190@KRBTEST.COM for kudu/127.29.112.189@KRBTEST.COM
I20260811 19:00:50.565753 20615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:50.528583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.189:33049 (local address 127.0.0.1:47094)
0811 19:00:50.528983 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:50.529012 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:50.529171 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:50.536590 (+ 7419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:50.536600 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:50.537576 (+ 976us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:50.537869 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:50.537886 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:50.541240 (+ 3354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:50.541249 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:50.543066 (+ 1817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:50.543113 (+ 47us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:50.545660 (+ 2547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:50.545669 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:50.545945 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:50.547073 (+ 1128us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:50.547101 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:50.550123 (+ 3022us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:00:50.560448 (+ 10325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:50.560461 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:50.560469 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:50.561435 (+ 966us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:50.562405 (+ 970us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:50.562415 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:50.562422 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:50.562989 (+ 567us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:50.564553 (+ 1564us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:00:50.564559 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:00:50.564785 (+ 226us) client_negotiation.cc:769] Sending connection context
0811 19:00:50.564954 (+ 169us) client_negotiation.cc:240] Negotiation successful
0811 19:00:50.565365 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213}
I20260811 19:00:50.567806 20752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:50.530900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47094 (local address 127.29.112.189:33049)
0811 19:00:50.532286 (+ 1386us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:50.532308 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:50.532407 (+ 99us) server_negotiation.cc:407] Connection header received
0811 19:00:50.534720 (+ 2313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:50.534751 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:50.535662 (+ 911us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:50.536571 (+ 909us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:50.538727 (+ 2156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:50.540971 (+ 2244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:50.543533 (+ 2562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:50.545375 (+ 1842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:50.545689 (+ 314us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:50.550857 (+ 5168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:50.550876 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:50.550907 (+ 31us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:50.550990 (+ 83us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:50.559934 (+ 8944us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:50.561763 (+ 1829us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:50.561779 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:50.561797 (+ 18us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:50.562189 (+ 392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:50.563444 (+ 1255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:50.563449 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:50.563452 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:50.564360 (+ 908us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:50.564613 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:50.565268 (+ 655us) server_negotiation.cc:299] Negotiation successful
0811 19:00:50.565808 (+ 540us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1057,"thread_start_us":695,"threads_started":1}
I20260811 19:00:50.573315 20610 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.112.188" port: 37973 } has no permanent_uuid. Determining permanent_uuid...
I20260811 19:00:50.575345 20615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:50.574787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.188:37973 (local address 127.0.0.1:46398)
0811 19:00:50.574995 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:50.575117 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.188:37973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260811 19:00:50.576854 20615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:50.576310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.188:37973 (local address 127.0.0.1:46406)
0811 19:00:50.576548 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:50.576682 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.188:37973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
W20260811 19:00:50.577214 20610 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.112.188:37973: Network error: Client connection negotiation failed: client connection to 127.29.112.188:37973: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.29.112.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.188@KRBTEST.COM" created.
Aug 11 19:00:50 dist-test-slave-h6n0 krb5kdc[20482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474850, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.189@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 19:00:50.595273 20755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:50.549484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44340 (local address 127.29.112.190:45847)
0811 19:00:50.565260 (+ 15776us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:50.565270 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:50.565305 (+ 35us) server_negotiation.cc:407] Connection header received
0811 19:00:50.565368 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:50.565375 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:50.565520 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:50.568301 (+ 2781us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:50.569637 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:50.572109 (+ 2472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:50.575450 (+ 3341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:50.576404 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:50.576571 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:50.586368 (+ 9797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:50.586374 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:50.586378 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:50.586420 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:50.591234 (+ 4814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:50.592215 (+ 981us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:50.592226 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:50.592229 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:50.592365 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:50.592776 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:50.592780 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:50.592782 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:50.593101 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:50.593442 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:50.594321 (+ 879us) server_negotiation.cc:299] Negotiation successful
0811 19:00:50.594712 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15368,"thread_start_us":16128,"threads_started":1}
I20260811 19:00:50.595317 20754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:50.549386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:45847 (local address 127.0.0.1:44340)
0811 19:00:50.563821 (+ 14435us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:50.563972 (+ 151us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:50.564464 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:50.565917 (+ 1453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:50.565931 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:50.568015 (+ 2084us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:50.568998 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:50.569026 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:50.572606 (+ 3580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:50.572618 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:50.575154 (+ 2536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:50.575188 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:50.576939 (+ 1751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:50.576951 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:50.577273 (+ 322us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:50.578142 (+ 869us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:50.578164 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:50.586042 (+ 7878us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:00:50.591458 (+ 5416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:50.591480 (+ 22us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:50.591487 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:50.592059 (+ 572us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:50.592486 (+ 427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:50.592490 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:50.592498 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:50.592646 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:50.593417 (+ 771us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:00:50.593432 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:00:50.593791 (+ 359us) client_negotiation.cc:769] Sending connection context
0811 19:00:50.594104 (+ 313us) client_negotiation.cc:240] Negotiation successful
0811 19:00:50.594713 (+ 609us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13703,"thread_start_us":13475,"threads_started":1}
I20260811 19:00:50.597957 20750 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.112.189" port: 33049 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 19:00:50 dist-test-slave-h6n0 krb5kdc[20482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474850, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.189@KRBTEST.COM for kudu/127.29.112.189@KRBTEST.COM
I20260811 19:00:50.615496 20754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:50.599739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.189:33049 (local address 127.0.0.1:47104)
0811 19:00:50.599993 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:50.600015 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:50.600420 (+ 405us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:50.600794 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:50.600798 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:50.601524 (+ 726us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:50.601776 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:50.601788 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:50.603508 (+ 1720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:50.603515 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:50.604554 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:50.604570 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:50.605347 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:50.605352 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:50.605581 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:50.606251 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:50.606270 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:50.608596 (+ 2326us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:00:50.611722 (+ 3126us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:50.611729 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:50.611733 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:50.612164 (+ 431us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:50.612539 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:50.612544 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:50.612548 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:50.612692 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:50.613521 (+ 829us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:00:50.613535 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:00:50.613797 (+ 262us) client_negotiation.cc:769] Sending connection context
0811 19:00:50.613982 (+ 185us) client_negotiation.cc:240] Negotiation successful
0811 19:00:50.614771 (+ 789us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"mutex_wait_us":173}
I20260811 19:00:50.615561 20752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:50.599483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47104 (local address 127.29.112.189:33049)
0811 19:00:50.599725 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:50.599731 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:50.600134 (+ 403us) server_negotiation.cc:407] Connection header received
0811 19:00:50.600580 (+ 446us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:50.600586 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:50.600660 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:50.600777 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:50.601964 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:50.603161 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:50.604743 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:50.605171 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:50.605326 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:50.608797 (+ 3471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:50.608802 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:50.608805 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:50.608835 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:50.611563 (+ 2728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:50.612289 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:50.612296 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:50.612300 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:50.612428 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:50.612799 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:50.612806 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:50.612809 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:50.613130 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:50.613622 (+ 492us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:50.614075 (+ 453us) server_negotiation.cc:299] Negotiation successful
0811 19:00:50.614885 (+ 810us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":372,"server-negotiator.queue_time_us":75}
I20260811 19:00:50.617411 20750 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.112.188" port: 37973 } has no permanent_uuid. Determining permanent_uuid...
I20260811 19:00:50.618948 20754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:50.618326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.188:37973 (local address 127.0.0.1:46422)
0811 19:00:50.618581 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:50.618749 (+ 168us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.188:37973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
W20260811 19:00:50.619758 20653 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.112.188:37973: connect: Connection refused (error 111)
I20260811 19:00:50.628496 20610 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.112.188" port: 37973 } attempt: 1
I20260811 19:00:50.629726 20615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:50.629223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.188:37973 (local address 127.0.0.1:46424)
0811 19:00:50.629442 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:50.629544 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.188:37973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260811 19:00:50.631523 20754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:50.630728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.188:37973 (local address 127.0.0.1:46434)
0811 19:00:50.631181 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:50.631322 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.188:37973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":273}
W20260811 19:00:50.631839 20750 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.112.188:37973: Network error: Client connection negotiation failed: client connection to 127.29.112.188:37973: connect: Connection refused (error 111)
I20260811 19:00:50.632695 20615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:50.631531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.188:37973 (local address 127.0.0.1:46446)
0811 19:00:50.632379 (+ 848us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:50.632495 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.188:37973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":683}
W20260811 19:00:50.632994 20610 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.112.188:37973: Network error: Client connection negotiation failed: client connection to 127.29.112.188:37973: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.188.keytab.
Entry for principal kudu/127.29.112.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.188.keytab.
W20260811 19:00:50.646507 30146 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.29.112.188: real 0.116s user 0.003s sys 0.026s
I20260811 19:00:50.647537 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474615205622-30146-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474615205622-30146-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474615205622-30146-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474615205622-30146-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.188:37973
--webserver_interface=127.29.112.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.188.keytab
--principal=kudu/127.29.112.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:40535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.190:45847,127.29.112.189:33049,127.29.112.188:37973
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 19:00:50.681334 20750 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.112.188" port: 37973 } attempt: 1
I20260811 19:00:50.682834 20754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:50.682239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.188:37973 (local address 127.0.0.1:46458)
0811 19:00:50.682488 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:50.682644 (+ 156us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.188:37973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260811 19:00:50.684247 20754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:50.683763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.188:37973 (local address 127.0.0.1:46470)
0811 19:00:50.684014 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:50.684110 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.188:37973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
W20260811 19:00:50.684689 20750 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.112.188:37973: Network error: Client connection negotiation failed: client connection to 127.29.112.188:37973: connect: Connection refused (error 111)
I20260811 19:00:50.701303 20610 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.112.188" port: 37973 } attempt: 2
I20260811 19:00:50.703312 20615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:50.702387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.188:37973 (local address 127.0.0.1:46486)
0811 19:00:50.702995 (+ 608us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:50.703119 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.188:37973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":410}
I20260811 19:00:50.705299 20615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:50.704605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.188:37973 (local address 127.0.0.1:46498)
0811 19:00:50.704864 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:50.704983 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.188:37973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
W20260811 19:00:50.705600 20610 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.112.188:37973: Network error: Client connection negotiation failed: client connection to 127.29.112.188:37973: connect: Connection refused (error 111)
I20260811 19:00:50.753162 20750 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.112.188" port: 37973 } attempt: 2
I20260811 19:00:50.754901 20754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:50.754218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.188:37973 (local address 127.0.0.1:46508)
0811 19:00:50.754490 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:50.754692 (+ 202us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.188:37973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115}
I20260811 19:00:50.756398 20754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:50.755962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.188:37973 (local address 127.0.0.1:46516)
0811 19:00:50.756147 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:50.756262 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.188:37973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260811 19:00:50.756628 20750 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.112.188:37973: Network error: Client connection negotiation failed: client connection to 127.29.112.188:37973: connect: Connection refused (error 111)
I20260811 19:00:50.811883 20610 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.112.188" port: 37973 } attempt: 3
I20260811 19:00:50.813750 20615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:50.813064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.188:37973 (local address 127.0.0.1:46526)
0811 19:00:50.813383 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:50.813509 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.188:37973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126}
I20260811 19:00:50.815023 20615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:50.814622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.188:37973 (local address 127.0.0.1:46538)
0811 19:00:50.814813 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:50.814891 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.188:37973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
W20260811 19:00:50.815236 20610 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.112.188:37973: Network error: Client connection negotiation failed: client connection to 127.29.112.188:37973: connect: Connection refused (error 111)
I20260811 19:00:50.847955 20750 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.112.188" port: 37973 } attempt: 3
I20260811 19:00:50.849386 20754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:50.848842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.188:37973 (local address 127.0.0.1:46554)
0811 19:00:50.849090 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:50.849195 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.188:37973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260811 19:00:50.850644 20754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:50.850094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.188:37973 (local address 127.0.0.1:46556)
0811 19:00:50.850351 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:50.850426 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.188:37973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143}
W20260811 19:00:50.850848 20750 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.112.188:37973: Network error: Client connection negotiation failed: client connection to 127.29.112.188:37973: connect: Connection refused (error 111)
I20260811 19:00:50.992527 20610 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.112.188" port: 37973 } attempt: 4
I20260811 19:00:50.993871 20615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:50.993364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.188:37973 (local address 127.0.0.1:46566)
0811 19:00:50.993600 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:50.993700 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.188:37973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260811 19:00:50.994083 20750 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.112.188" port: 37973 } attempt: 4
I20260811 19:00:50.995249 20754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:50.994766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.188:37973 (local address 127.0.0.1:46568)
0811 19:00:50.994987 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:50.995102 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.188:37973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260811 19:00:50.995491 20615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:50.994964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.188:37973 (local address 127.0.0.1:46580)
0811 19:00:50.995182 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:50.995295 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.188:37973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260811 19:00:50.995787 20610 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.112.188:37973: Network error: Client connection negotiation failed: client connection to 127.29.112.188:37973: connect: Connection refused (error 111)
I20260811 19:00:50.996304 20754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:50.995990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.188:37973 (local address 127.0.0.1:46596)
0811 19:00:50.996118 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:50.996199 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.188:37973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260811 19:00:50.996644 20750 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.112.188:37973: Network error: Client connection negotiation failed: client connection to 127.29.112.188:37973: connect: Connection refused (error 111)
W20260811 19:00:51.117219 20760 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:00:51.117637 20760 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:00:51.117807 20760 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:00:51.125970 20760 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:00:51.126185 20760 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:00:51.126614 20760 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:00:51.126873 20760 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:00:51.127115 20760 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:00:51.139835 20760 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:40535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474615205622-30146-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474615205622-30146-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.29.112.190:45847,127.29.112.189:33049,127.29.112.188:37973
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.188.keytab
--principal=kudu/127.29.112.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.188:37973
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474615205622-30146-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.29.112.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474615205622-30146-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:00:51.141371 20760 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:00:51.142825 20760 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:00:51.158214 20768 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:51.158725 20769 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:51.158725 20772 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:51.160537 20760 server_base.cc:1034] running on GCE node
I20260811 19:00:51.161998 20760 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:00:51.164655 20760 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:00:51.166136 20760 hybrid_clock.cc:648] HybridClock initialized: now 1786474851166006 us; error 145 us; skew 500 ppm
I20260811 19:00:51.166729 20760 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:00:51 dist-test-slave-h6n0 krb5kdc[20482](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474851, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:00:51.171934 20760 init.cc:377] Logged in from keytab as kudu/127.29.112.188@KRBTEST.COM (short username kudu)
I20260811 19:00:51.174085 20760 webserver.cc:460] Webserver started at http://127.29.112.188:45943/ using document root <none> and password file <none>
I20260811 19:00:51.175356 20760 fs_manager.cc:359] Metadata directory not provided
I20260811 19:00:51.175704 20760 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:00:51.176241 20760 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:00:51.181492 20760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474615205622-30146-0/minicluster-data/master-2/data/instance:
uuid: "9db07befbc1d45c0933bf9687bec1e5f"
format_stamp: "Formatted at 2026-08-11 19:00:51 on dist-test-slave-h6n0"
server_key: "c389c8d09073e112867810e9bbd4ab17"
server_key_iv: "d6464d2a6483df77d254fcde3378bff4"
server_key_version: "encryptionkey@0"
I20260811 19:00:51.182792 20760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474615205622-30146-0/minicluster-data/master-2/wal/instance:
uuid: "9db07befbc1d45c0933bf9687bec1e5f"
format_stamp: "Formatted at 2026-08-11 19:00:51 on dist-test-slave-h6n0"
server_key: "c389c8d09073e112867810e9bbd4ab17"
server_key_iv: "d6464d2a6483df77d254fcde3378bff4"
server_key_version: "encryptionkey@0"
I20260811 19:00:51.190922 20760 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260811 19:00:51.195976 20781 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:51.198658 20760 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.006s sys 0.000s
I20260811 19:00:51.198982 20760 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474615205622-30146-0/minicluster-data/master-2/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474615205622-30146-0/minicluster-data/master-2/wal
uuid: "9db07befbc1d45c0933bf9687bec1e5f"
format_stamp: "Formatted at 2026-08-11 19:00:51 on dist-test-slave-h6n0"
server_key: "c389c8d09073e112867810e9bbd4ab17"
server_key_iv: "d6464d2a6483df77d254fcde3378bff4"
server_key_version: "encryptionkey@0"
I20260811 19:00:51.199321 20760 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474615205622-30146-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474615205622-30146-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474615205622-30146-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:00:51.217118 20760 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:00:51.224269 20760 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:00:51.224864 20760 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:00:51.252377 20760 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.188:37973
I20260811 19:00:51.252560 20865 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.188:37973 every 8 connection(s)
I20260811 19:00:51.254040 20760 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474615205622-30146-0/minicluster-data/master-2/data/info.pb
I20260811 19:00:51.258893 20867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:00:51.263516 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 20760
I20260811 19:00:51.263779 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474615205622-30146-0/minicluster-data/master-2/wal/instance
I20260811 19:00:51.264441 30146 external_mini_cluster.cc:1428] Setting key e9a3e2faba59cb38ac523ac391fe813d
I20260811 19:00:51.268707 20867 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.112.190" port: 45847 } has no permanent_uuid. Determining permanent_uuid...
I20260811 19:00:51.273132 20610 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.112.188" port: 37973 } attempt: 5
Aug 11 19:00:51 dist-test-slave-h6n0 krb5kdc[20482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474848, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.188@KRBTEST.COM
Aug 11 19:00:51 dist-test-slave-h6n0 krb5kdc[20482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474849, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.190@KRBTEST.COM for kudu/127.29.112.188@KRBTEST.COM
Aug 11 19:00:51 dist-test-slave-h6n0 krb5kdc[20482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474851, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.188@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 19:00:51.295892 20750 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.112.188" port: 37973 } attempt: 5
I20260811 19:00:51.297834 20615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:51.273832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.188:37973 (local address 127.0.0.1:46616)
0811 19:00:51.274262 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:51.274289 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:51.274460 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:51.276677 (+ 2217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:51.276684 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:51.277251 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:51.277490 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:51.277506 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:51.279270 (+ 1764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:51.279277 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:51.280838 (+ 1561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:51.280856 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:51.282318 (+ 1462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:51.282325 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:51.282653 (+ 328us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:51.283840 (+ 1187us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:51.283867 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:51.287125 (+ 3258us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:00:51.292887 (+ 5762us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:51.292896 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:51.292903 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:51.294092 (+ 1189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:51.294828 (+ 736us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:51.294834 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:51.294838 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:51.295072 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:51.296360 (+ 1288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:00:51.296366 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:00:51.296760 (+ 394us) client_negotiation.cc:769] Sending connection context
0811 19:00:51.297056 (+ 296us) client_negotiation.cc:240] Negotiation successful
0811 19:00:51.297388 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279}
I20260811 19:00:51.299530 20874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:51.275624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46616 (local address 127.29.112.188:37973)
0811 19:00:51.276308 (+ 684us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:51.276315 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:51.276357 (+ 42us) server_negotiation.cc:407] Connection header received
0811 19:00:51.276410 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:51.276415 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:51.276487 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:51.276635 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:51.277783 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:51.279097 (+ 1314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:51.281027 (+ 1930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:51.282028 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:51.282195 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:51.287442 (+ 5247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:51.287455 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:51.287463 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:51.287508 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:51.292656 (+ 5148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:51.294318 (+ 1662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:51.294324 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:51.294328 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:51.294662 (+ 334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:51.295214 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:51.295220 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:51.295223 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:51.295699 (+ 476us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:51.295894 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:51.298169 (+ 2275us) server_negotiation.cc:299] Negotiation successful
0811 19:00:51.298431 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":85,"server-negotiator.queue_time_us":545,"thread_start_us":361,"threads_started":1}
I20260811 19:00:51.299530 20870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:51.267191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46600 (local address 127.29.112.188:37973)
0811 19:00:51.268015 (+ 824us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:51.268027 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:51.268087 (+ 60us) server_negotiation.cc:407] Connection header received
0811 19:00:51.270136 (+ 2049us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:51.270162 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:51.270767 (+ 605us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:51.271408 (+ 641us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:51.273645 (+ 2237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:51.276486 (+ 2841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:51.278603 (+ 2117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:51.279797 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:51.280021 (+ 224us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:51.285773 (+ 5752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:51.285784 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:51.285798 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:51.285830 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:51.291404 (+ 5574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:51.293380 (+ 1976us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:51.293393 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:51.293406 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:51.293684 (+ 278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:51.294301 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:51.294310 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:51.294345 (+ 35us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:51.295379 (+ 1034us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:51.295717 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:51.297089 (+ 1372us) server_negotiation.cc:299] Negotiation successful
0811 19:00:51.297600 (+ 511us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":608,"thread_start_us":378,"threads_started":1}
I20260811 19:00:51.301023 20872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:51.273830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44344 (local address 127.29.112.190:45847)
0811 19:00:51.276060 (+ 2230us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:51.276067 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:51.276088 (+ 21us) server_negotiation.cc:407] Connection header received
0811 19:00:51.276149 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:51.276153 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:51.276210 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:51.276350 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:51.278298 (+ 1948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:51.279517 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:51.282044 (+ 2527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:51.283013 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:51.283159 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:51.290025 (+ 6866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:51.290039 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:51.290048 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:51.290216 (+ 168us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:51.296485 (+ 6269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:51.297494 (+ 1009us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:51.297500 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:51.297516 (+ 16us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:51.297701 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:51.298468 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:51.298477 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:51.298483 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:51.299000 (+ 517us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:51.299232 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:51.300092 (+ 860us) server_negotiation.cc:299] Negotiation successful
0811 19:00:51.300444 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2062,"thread_start_us":1898,"threads_started":1}
I20260811 19:00:51.302580 20873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:51.273924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:45847 (local address 127.0.0.1:44344)
0811 19:00:51.275264 (+ 1340us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:51.275327 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:51.275504 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:51.276427 (+ 923us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:51.276435 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:51.277403 (+ 968us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:51.278103 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:51.278117 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:51.279744 (+ 1627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:51.279750 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:51.281739 (+ 1989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:51.281771 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:51.283278 (+ 1507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:51.283288 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:51.283447 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:51.284090 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:51.284116 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:51.289530 (+ 5414us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:00:51.296655 (+ 7125us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:51.296668 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:51.296675 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:51.297347 (+ 672us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:51.297881 (+ 534us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:51.297889 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:51.297894 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:51.298279 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:51.299354 (+ 1075us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:00:51.299366 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:00:51.299643 (+ 277us) client_negotiation.cc:769] Sending connection context
0811 19:00:51.301366 (+ 1723us) client_negotiation.cc:240] Negotiation successful
0811 19:00:51.301870 (+ 504us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1044,"mutex_wait_us":181,"thread_start_us":828,"threads_started":1}
I20260811 19:00:51.312177 20867 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.112.189" port: 33049 } has no permanent_uuid. Determining permanent_uuid...
I20260811 19:00:51.313956 20610 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 312aa2c1b36b44998c5e3e4f77f36d88: Bootstrap starting.
Aug 11 19:00:51 dist-test-slave-h6n0 krb5kdc[20482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474850, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.189@KRBTEST.COM for kudu/127.29.112.188@KRBTEST.COM
I20260811 19:00:51.319393 20610 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 312aa2c1b36b44998c5e3e4f77f36d88: Neither blocks nor log segments found. Creating new log.
I20260811 19:00:51.321164 20610 log.cc:824] T 00000000000000000000000000000000 P 312aa2c1b36b44998c5e3e4f77f36d88: Log is configured to *not* fsync() on all Append() calls
I20260811 19:00:51.326617 20754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:51.296609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.188:37973 (local address 127.0.0.1:46626)
0811 19:00:51.297030 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:51.297050 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:51.297169 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:51.301163 (+ 3994us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:51.301170 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:51.301969 (+ 799us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:51.302235 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:51.302255 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:51.305035 (+ 2780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:51.305044 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:51.307604 (+ 2560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:51.307639 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:51.310904 (+ 3265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:51.310913 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:51.311142 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:51.312265 (+ 1123us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:51.312304 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:51.315807 (+ 3503us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:00:51.321086 (+ 5279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:51.321093 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:51.321097 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:51.321747 (+ 650us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:51.322609 (+ 862us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:51.322618 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:51.322625 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:51.322885 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:51.323893 (+ 1008us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:00:51.323902 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:00:51.324204 (+ 302us) client_negotiation.cc:769] Sending connection context
0811 19:00:51.324792 (+ 588us) client_negotiation.cc:240] Negotiation successful
0811 19:00:51.325481 (+ 689us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281}
Aug 11 19:00:51 dist-test-slave-h6n0 krb5kdc[20482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474848, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 19:00:51.330586 20870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:51.297383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46626 (local address 127.29.112.188:37973)
0811 19:00:51.300689 (+ 3306us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:51.300696 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:51.300718 (+ 22us) server_negotiation.cc:407] Connection header received
0811 19:00:51.300788 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:51.300794 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:51.300887 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:51.303044 (+ 2157us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:51.303204 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:51.304701 (+ 1497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:51.307924 (+ 3223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:51.310639 (+ 2715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:51.311805 (+ 1166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:51.316168 (+ 4363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:51.316180 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:51.316186 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:51.316240 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:51.320826 (+ 4586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:51.322135 (+ 1309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:51.322145 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:51.322152 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:51.322382 (+ 230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:51.323118 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:51.323127 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:51.323133 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:51.323687 (+ 554us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:51.328598 (+ 4911us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:51.329002 (+ 404us) server_negotiation.cc:299] Negotiation successful
0811 19:00:51.329630 (+ 628us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3153,"thread_start_us":7265,"threads_started":1}
I20260811 19:00:51.338165 20610 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 312aa2c1b36b44998c5e3e4f77f36d88: No bootstrap required, opened a new log
Aug 11 19:00:51 dist-test-slave-h6n0 krb5kdc[20482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474851, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.188@KRBTEST.COM for kudu/127.29.112.189@KRBTEST.COM
I20260811 19:00:51.339004 20750 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6969cb95acbc49fa98783fb8853482c9: Bootstrap starting.
I20260811 19:00:51.339571 20872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:51.308776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44352 (local address 127.29.112.190:45847)
0811 19:00:51.309261 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:51.309267 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:51.309923 (+ 656us) server_negotiation.cc:407] Connection header received
0811 19:00:51.310344 (+ 421us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:51.310350 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:51.310659 (+ 309us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:51.310904 (+ 245us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:51.313958 (+ 3054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:51.315385 (+ 1427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:51.318506 (+ 3121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:51.319567 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:51.319740 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:51.328983 (+ 9243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:51.328993 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:51.328999 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:51.329057 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:51.334608 (+ 5551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:51.335731 (+ 1123us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:51.335738 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:51.335742 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:51.335929 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:51.336815 (+ 886us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:51.336834 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:51.336838 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:51.337228 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:51.337540 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:51.338941 (+ 1401us) server_negotiation.cc:299] Negotiation successful
0811 19:00:51.339223 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":280}
I20260811 19:00:51.344442 20750 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6969cb95acbc49fa98783fb8853482c9: Neither blocks nor log segments found. Creating new log.
I20260811 19:00:51.346849 20750 log.cc:824] T 00000000000000000000000000000000 P 6969cb95acbc49fa98783fb8853482c9: Log is configured to *not* fsync() on all Append() calls
I20260811 19:00:51.354333 20750 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6969cb95acbc49fa98783fb8853482c9: No bootstrap required, opened a new log
I20260811 19:00:51.354776 20873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:51.313649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.189:33049 (local address 127.0.0.1:47116)
0811 19:00:51.314742 (+ 1093us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:51.314766 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:51.314918 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:51.327646 (+ 12728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:51.327656 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:51.328318 (+ 662us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:51.328647 (+ 329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:51.328661 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:51.330680 (+ 2019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:51.330689 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:51.332492 (+ 1803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:51.332512 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:51.335486 (+ 2974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:51.335495 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:51.335769 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:51.336548 (+ 779us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:51.336574 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:51.340276 (+ 3702us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:00:51.348325 (+ 8049us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:51.348336 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:51.348341 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:51.349369 (+ 1028us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:51.351603 (+ 2234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:51.351610 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:51.351616 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:51.352034 (+ 418us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:51.353222 (+ 1188us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:00:51.353229 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:00:51.353495 (+ 266us) client_negotiation.cc:769] Sending connection context
0811 19:00:51.353664 (+ 169us) client_negotiation.cc:240] Negotiation successful
0811 19:00:51.354093 (+ 429us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":937}
I20260811 19:00:51.355085 20610 raft_consensus.cc:348] T 00000000000000000000000000000000 P 312aa2c1b36b44998c5e3e4f77f36d88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "312aa2c1b36b44998c5e3e4f77f36d88" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 45847 } } peers { permanent_uuid: "6969cb95acbc49fa98783fb8853482c9" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 33049 } } peers { permanent_uuid: "9db07befbc1d45c0933bf9687bec1e5f" member_type: VOTER last_known_addr { host: "127.29.112.188" port: 37973 } }
I20260811 19:00:51.355581 20610 raft_consensus.cc:374] T 00000000000000000000000000000000 P 312aa2c1b36b44998c5e3e4f77f36d88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:00:51.355726 20610 raft_consensus.cc:729] T 00000000000000000000000000000000 P 312aa2c1b36b44998c5e3e4f77f36d88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 312aa2c1b36b44998c5e3e4f77f36d88, State: Initialized, Role: FOLLOWER
I20260811 19:00:51.357223 20610 consensus_queue.cc:260] T 00000000000000000000000000000000 P 312aa2c1b36b44998c5e3e4f77f36d88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "312aa2c1b36b44998c5e3e4f77f36d88" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 45847 } } peers { permanent_uuid: "6969cb95acbc49fa98783fb8853482c9" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 33049 } } peers { permanent_uuid: "9db07befbc1d45c0933bf9687bec1e5f" member_type: VOTER last_known_addr { host: "127.29.112.188" port: 37973 } }
I20260811 19:00:51.357851 20877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:51.314290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47116 (local address 127.29.112.189:33049)
0811 19:00:51.326803 (+ 12513us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:51.326811 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:51.326838 (+ 27us) server_negotiation.cc:407] Connection header received
0811 19:00:51.326907 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:51.326913 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:51.327180 (+ 267us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:51.328835 (+ 1655us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:51.329022 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:51.330437 (+ 1415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:51.333063 (+ 2626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:51.335232 (+ 2169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:51.335447 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:51.340551 (+ 5104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:51.340560 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:51.340567 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:51.340618 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:51.347652 (+ 7034us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:51.351040 (+ 3388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:51.351048 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:51.351057 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:51.351327 (+ 270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:51.352329 (+ 1002us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:51.352336 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:51.352340 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:51.353004 (+ 664us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:51.355945 (+ 2941us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:51.356686 (+ 741us) server_negotiation.cc:299] Negotiation successful
0811 19:00:51.357042 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12309,"thread_start_us":13056,"threads_started":1}
I20260811 19:00:51.359721 20867 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.112.188" port: 37973 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 19:00:51 dist-test-slave-h6n0 krb5kdc[20482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474848, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.189@KRBTEST.COM
I20260811 19:00:51.371407 20886 sys_catalog.cc:455] T 00000000000000000000000000000000 P 312aa2c1b36b44998c5e3e4f77f36d88 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "312aa2c1b36b44998c5e3e4f77f36d88" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 45847 } } peers { permanent_uuid: "6969cb95acbc49fa98783fb8853482c9" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 33049 } } peers { permanent_uuid: "9db07befbc1d45c0933bf9687bec1e5f" member_type: VOTER last_known_addr { host: "127.29.112.188" port: 37973 } } }
I20260811 19:00:51.372162 20886 sys_catalog.cc:458] T 00000000000000000000000000000000 P 312aa2c1b36b44998c5e3e4f77f36d88 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 19:00:51.373651 20610 sys_catalog.cc:564] T 00000000000000000000000000000000 P 312aa2c1b36b44998c5e3e4f77f36d88 [sys.catalog]: configured and running, proceeding with master startup.
Aug 11 19:00:51 dist-test-slave-h6n0 krb5kdc[20482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474851, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.188@KRBTEST.COM for kudu/127.29.112.188@KRBTEST.COM
I20260811 19:00:51.375926 20750 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6969cb95acbc49fa98783fb8853482c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "312aa2c1b36b44998c5e3e4f77f36d88" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 45847 } } peers { permanent_uuid: "6969cb95acbc49fa98783fb8853482c9" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 33049 } } peers { permanent_uuid: "9db07befbc1d45c0933bf9687bec1e5f" member_type: VOTER last_known_addr { host: "127.29.112.188" port: 37973 } }
I20260811 19:00:51.376431 20750 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6969cb95acbc49fa98783fb8853482c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:00:51.376573 20750 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6969cb95acbc49fa98783fb8853482c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6969cb95acbc49fa98783fb8853482c9, State: Initialized, Role: FOLLOWER
I20260811 19:00:51.377436 20750 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6969cb95acbc49fa98783fb8853482c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "312aa2c1b36b44998c5e3e4f77f36d88" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 45847 } } peers { permanent_uuid: "6969cb95acbc49fa98783fb8853482c9" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 33049 } } peers { permanent_uuid: "9db07befbc1d45c0933bf9687bec1e5f" member_type: VOTER last_known_addr { host: "127.29.112.188" port: 37973 } }
I20260811 19:00:51.378672 20883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:51.341872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47122 (local address 127.29.112.189:33049)
0811 19:00:51.348677 (+ 6805us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:51.348684 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:51.349440 (+ 756us) server_negotiation.cc:407] Connection header received
0811 19:00:51.349538 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:51.349547 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:51.349666 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:51.349848 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:51.352743 (+ 2895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:51.354334 (+ 1591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:51.356485 (+ 2151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:51.357319 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:51.357452 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:51.364969 (+ 7517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:51.364981 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:51.364987 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:51.365067 (+ 80us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:51.372448 (+ 7381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:51.373814 (+ 1366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:51.373821 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:51.373825 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:51.374113 (+ 288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:51.374823 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:51.374898 (+ 75us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:51.374903 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:51.375499 (+ 596us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:51.375864 (+ 365us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:51.377395 (+ 1531us) server_negotiation.cc:299] Negotiation successful
0811 19:00:51.377683 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6646,"thread_start_us":6437,"threads_started":1}
I20260811 19:00:51.384979 20891 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6969cb95acbc49fa98783fb8853482c9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "312aa2c1b36b44998c5e3e4f77f36d88" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 45847 } } peers { permanent_uuid: "6969cb95acbc49fa98783fb8853482c9" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 33049 } } peers { permanent_uuid: "9db07befbc1d45c0933bf9687bec1e5f" member_type: VOTER last_known_addr { host: "127.29.112.188" port: 37973 } } }
I20260811 19:00:51.385941 20891 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6969cb95acbc49fa98783fb8853482c9 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 19:00:51.387849 30146 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 19:00:51.389274 20870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:51.360938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46636 (local address 127.29.112.188:37973)
0811 19:00:51.361151 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:51.361157 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:51.361184 (+ 27us) server_negotiation.cc:407] Connection header received
0811 19:00:51.361242 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:51.361249 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:51.361338 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:51.361450 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:51.363211 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:51.365615 (+ 2404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:51.368911 (+ 3296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:51.370024 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:51.370283 (+ 259us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:51.377140 (+ 6857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:51.377150 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:51.377159 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:51.377209 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:51.382377 (+ 5168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:51.383550 (+ 1173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:51.383558 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:51.383564 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:51.383868 (+ 304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:51.384894 (+ 1026us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:51.384903 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:51.384910 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:51.386675 (+ 1765us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:51.387042 (+ 367us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:51.388350 (+ 1308us) server_negotiation.cc:299] Negotiation successful
0811 19:00:51.388797 (+ 447us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 19:00:51.389390 20750 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6969cb95acbc49fa98783fb8853482c9 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:00:51.391604 20873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:51.360638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.188:37973 (local address 127.0.0.1:46636)
0811 19:00:51.360906 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:51.360936 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:51.361113 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:51.361518 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:51.361527 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:51.362292 (+ 765us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:51.362684 (+ 392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:51.362705 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:51.365956 (+ 3251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:51.365966 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:51.368595 (+ 2629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:51.368637 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:51.370786 (+ 2149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:51.370793 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:51.370991 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:51.371954 (+ 963us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:51.371982 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:51.376840 (+ 4858us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:00:51.382582 (+ 5742us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:51.382591 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:51.382598 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:51.383372 (+ 774us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:51.384083 (+ 711us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:51.384091 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:51.384098 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:51.384533 (+ 435us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:51.387134 (+ 2601us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:00:51.387144 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:00:51.387653 (+ 509us) client_negotiation.cc:769] Sending connection context
0811 19:00:51.387941 (+ 288us) client_negotiation.cc:240] Negotiation successful
0811 19:00:51.390749 (+ 2808us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71}
I20260811 19:00:51.402163 20867 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9db07befbc1d45c0933bf9687bec1e5f: Bootstrap starting.
I20260811 19:00:51.409616 20867 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9db07befbc1d45c0933bf9687bec1e5f: Neither blocks nor log segments found. Creating new log.
I20260811 19:00:51.416631 20867 log.cc:824] T 00000000000000000000000000000000 P 9db07befbc1d45c0933bf9687bec1e5f: Log is configured to *not* fsync() on all Append() calls
I20260811 19:00:51.429992 20867 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9db07befbc1d45c0933bf9687bec1e5f: No bootstrap required, opened a new log
W20260811 19:00:51.431114 20914 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 312aa2c1b36b44998c5e3e4f77f36d88: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 19:00:51.431588 20914 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 19:00:51.407788 20497 master_runner.cc:186] Error getting master registration for 127.29.112.189:33049: OK, instance_id { permanent_uuid: "6969cb95acbc49fa98783fb8853482c9" instance_seqno: 1786474850511034 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 19:00:51.446879 20497 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 19:00:51.450443 20883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:51.414825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47130 (local address 127.29.112.189:33049)
0811 19:00:51.415102 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:51.415112 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:51.416034 (+ 922us) server_negotiation.cc:407] Connection header received
0811 19:00:51.416464 (+ 430us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:51.416471 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:51.416587 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:51.416697 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:51.420490 (+ 3793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:51.421919 (+ 1429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:51.426687 (+ 4768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:51.427536 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:51.427684 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:51.435630 (+ 7946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:51.435637 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:51.435643 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:51.435700 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:51.440024 (+ 4324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:51.442658 (+ 2634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:51.442665 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:51.442672 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:51.442890 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:51.446727 (+ 3837us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:51.446740 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:51.446747 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:51.447492 (+ 745us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:51.448343 (+ 851us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:51.449450 (+ 1107us) server_negotiation.cc:299] Negotiation successful
0811 19:00:51.450057 (+ 607us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 19:00:51.451345 20886 raft_consensus.cc:482] T 00000000000000000000000000000000 P 312aa2c1b36b44998c5e3e4f77f36d88 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:00:51.452324 20886 raft_consensus.cc:504] T 00000000000000000000000000000000 P 312aa2c1b36b44998c5e3e4f77f36d88 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "312aa2c1b36b44998c5e3e4f77f36d88" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 45847 } } peers { permanent_uuid: "6969cb95acbc49fa98783fb8853482c9" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 33049 } } peers { permanent_uuid: "9db07befbc1d45c0933bf9687bec1e5f" member_type: VOTER last_known_addr { host: "127.29.112.188" port: 37973 } }
I20260811 19:00:51.453935 20872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:51.415311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44368 (local address 127.29.112.190:45847)
0811 19:00:51.415567 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:51.415574 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:51.422873 (+ 7299us) server_negotiation.cc:407] Connection header received
0811 19:00:51.426100 (+ 3227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:51.426113 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:51.426394 (+ 281us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:51.426582 (+ 188us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:51.433119 (+ 6537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:51.434905 (+ 1786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:51.437632 (+ 2727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:51.438450 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:51.438605 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:51.442886 (+ 4281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:51.442895 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:51.442901 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:51.442966 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:51.448594 (+ 5628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:51.450044 (+ 1450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:51.450051 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:51.450055 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:51.450290 (+ 235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:51.451231 (+ 941us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:51.451251 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:51.451258 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:51.451838 (+ 580us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:51.452309 (+ 471us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:51.453048 (+ 739us) server_negotiation.cc:299] Negotiation successful
0811 19:00:51.453414 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 19:00:51.455679 20692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "312aa2c1b36b44998c5e3e4f77f36d88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6969cb95acbc49fa98783fb8853482c9" is_pre_election: true
I20260811 19:00:51.456449 20692 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6969cb95acbc49fa98783fb8853482c9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 312aa2c1b36b44998c5e3e4f77f36d88 in term 0.
I20260811 19:00:51.457170 20526 leader_election.cc:302] T 00000000000000000000000000000000 P 312aa2c1b36b44998c5e3e4f77f36d88 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 312aa2c1b36b44998c5e3e4f77f36d88, 6969cb95acbc49fa98783fb8853482c9; no voters:
I20260811 19:00:51.458261 20931 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 312aa2c1b36b44998c5e3e4f77f36d88 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 19:00:51.458462 20931 raft_consensus.cc:482] T 00000000000000000000000000000000 P 312aa2c1b36b44998c5e3e4f77f36d88 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 19:00:51.458703 20931 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 312aa2c1b36b44998c5e3e4f77f36d88 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:00:51.458941 20886 leader_election.cc:288] T 00000000000000000000000000000000 P 312aa2c1b36b44998c5e3e4f77f36d88 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6969cb95acbc49fa98783fb8853482c9 (127.29.112.189:33049), 9db07befbc1d45c0933bf9687bec1e5f (127.29.112.188:37973)
I20260811 19:00:51.463575 20931 raft_consensus.cc:504] T 00000000000000000000000000000000 P 312aa2c1b36b44998c5e3e4f77f36d88 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "312aa2c1b36b44998c5e3e4f77f36d88" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 45847 } } peers { permanent_uuid: "6969cb95acbc49fa98783fb8853482c9" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 33049 } } peers { permanent_uuid: "9db07befbc1d45c0933bf9687bec1e5f" member_type: VOTER last_known_addr { host: "127.29.112.188" port: 37973 } }
I20260811 19:00:51.465731 20692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "312aa2c1b36b44998c5e3e4f77f36d88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6969cb95acbc49fa98783fb8853482c9"
I20260811 19:00:51.466064 20931 leader_election.cc:288] T 00000000000000000000000000000000 P 312aa2c1b36b44998c5e3e4f77f36d88 [CANDIDATE]: Term 1 election: Requested vote from peers 6969cb95acbc49fa98783fb8853482c9 (127.29.112.189:33049), 9db07befbc1d45c0933bf9687bec1e5f (127.29.112.188:37973)
I20260811 19:00:51.466084 20692 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6969cb95acbc49fa98783fb8853482c9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:00:51.474141 20867 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9db07befbc1d45c0933bf9687bec1e5f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "312aa2c1b36b44998c5e3e4f77f36d88" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 45847 } } peers { permanent_uuid: "6969cb95acbc49fa98783fb8853482c9" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 33049 } } peers { permanent_uuid: "9db07befbc1d45c0933bf9687bec1e5f" member_type: VOTER last_known_addr { host: "127.29.112.188" port: 37973 } }
I20260811 19:00:51.474328 20692 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6969cb95acbc49fa98783fb8853482c9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 312aa2c1b36b44998c5e3e4f77f36d88 in term 1.
I20260811 19:00:51.475032 20867 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9db07befbc1d45c0933bf9687bec1e5f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:00:51.475273 20867 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9db07befbc1d45c0933bf9687bec1e5f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9db07befbc1d45c0933bf9687bec1e5f, State: Initialized, Role: FOLLOWER
I20260811 19:00:51.475808 20526 leader_election.cc:302] T 00000000000000000000000000000000 P 312aa2c1b36b44998c5e3e4f77f36d88 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 312aa2c1b36b44998c5e3e4f77f36d88, 6969cb95acbc49fa98783fb8853482c9; no voters:
I20260811 19:00:51.476269 20931 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 312aa2c1b36b44998c5e3e4f77f36d88 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:00:51.476461 20931 raft_consensus.cc:686] T 00000000000000000000000000000000 P 312aa2c1b36b44998c5e3e4f77f36d88 [term 1 LEADER]: Becoming Leader. State: Replica: 312aa2c1b36b44998c5e3e4f77f36d88, State: Running, Role: LEADER
I20260811 19:00:51.476207 20867 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9db07befbc1d45c0933bf9687bec1e5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "312aa2c1b36b44998c5e3e4f77f36d88" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 45847 } } peers { permanent_uuid: "6969cb95acbc49fa98783fb8853482c9" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 33049 } } peers { permanent_uuid: "9db07befbc1d45c0933bf9687bec1e5f" member_type: VOTER last_known_addr { host: "127.29.112.188" port: 37973 } }
I20260811 19:00:51.477267 20931 consensus_queue.cc:237] T 00000000000000000000000000000000 P 312aa2c1b36b44998c5e3e4f77f36d88 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "312aa2c1b36b44998c5e3e4f77f36d88" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 45847 } } peers { permanent_uuid: "6969cb95acbc49fa98783fb8853482c9" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 33049 } } peers { permanent_uuid: "9db07befbc1d45c0933bf9687bec1e5f" member_type: VOTER last_known_addr { host: "127.29.112.188" port: 37973 } }
I20260811 19:00:51.482347 20870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:51.424049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46642 (local address 127.29.112.188:37973)
0811 19:00:51.424561 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:51.424571 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:51.435383 (+ 10812us) server_negotiation.cc:407] Connection header received
0811 19:00:51.441596 (+ 6213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:51.441603 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:51.441712 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:51.441787 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:51.454488 (+ 12701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:51.455632 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:51.457887 (+ 2255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:51.458604 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:51.458714 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:51.463505 (+ 4791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:51.463525 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:51.463534 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:51.463601 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:51.472017 (+ 8416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:51.475038 (+ 3021us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:51.475073 (+ 35us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:51.475079 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:51.475626 (+ 547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:51.477009 (+ 1383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:51.477015 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:51.477019 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:51.477951 (+ 932us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:51.478759 (+ 808us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:51.481036 (+ 2277us) server_negotiation.cc:299] Negotiation successful
0811 19:00:51.481607 (+ 571us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309}
I20260811 19:00:51.482359 20886 sys_catalog.cc:455] T 00000000000000000000000000000000 P 312aa2c1b36b44998c5e3e4f77f36d88 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 312aa2c1b36b44998c5e3e4f77f36d88. Latest consensus state: current_term: 1 leader_uuid: "312aa2c1b36b44998c5e3e4f77f36d88" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "312aa2c1b36b44998c5e3e4f77f36d88" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 45847 } } peers { permanent_uuid: "6969cb95acbc49fa98783fb8853482c9" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 33049 } } peers { permanent_uuid: "9db07befbc1d45c0933bf9687bec1e5f" member_type: VOTER last_known_addr { host: "127.29.112.188" port: 37973 } } }
I20260811 19:00:51.482765 20886 sys_catalog.cc:458] T 00000000000000000000000000000000 P 312aa2c1b36b44998c5e3e4f77f36d88 [sys.catalog]: This master's current role is: LEADER
W20260811 19:00:51.493810 20656 tablet.cc:2369] T 00000000000000000000000000000000 P 6969cb95acbc49fa98783fb8853482c9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 19:00:51.496984 20938 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:00:51.499519 20938 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:00:51.508637 20867 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9db07befbc1d45c0933bf9687bec1e5f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:00:51.508719 20826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "312aa2c1b36b44998c5e3e4f77f36d88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9db07befbc1d45c0933bf9687bec1e5f" is_pre_election: true
I20260811 19:00:51.509706 20825 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "312aa2c1b36b44998c5e3e4f77f36d88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9db07befbc1d45c0933bf9687bec1e5f"
I20260811 19:00:51.510108 20826 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9db07befbc1d45c0933bf9687bec1e5f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 312aa2c1b36b44998c5e3e4f77f36d88 in term 0.
I20260811 19:00:51.506305 20935 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9db07befbc1d45c0933bf9687bec1e5f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "312aa2c1b36b44998c5e3e4f77f36d88" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 45847 } } peers { permanent_uuid: "6969cb95acbc49fa98783fb8853482c9" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 33049 } } peers { permanent_uuid: "9db07befbc1d45c0933bf9687bec1e5f" member_type: VOTER last_known_addr { host: "127.29.112.188" port: 37973 } } }
I20260811 19:00:51.511793 20935 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9db07befbc1d45c0933bf9687bec1e5f [sys.catalog]: This master's current role is: FOLLOWER
W20260811 19:00:51.513525 20531 tablet.cc:2369] T 00000000000000000000000000000000 P 312aa2c1b36b44998c5e3e4f77f36d88: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 19:00:51.518314 20942 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 19:00:51.523082 20692 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6969cb95acbc49fa98783fb8853482c9 [term 1 FOLLOWER]: Refusing update from remote peer 312aa2c1b36b44998c5e3e4f77f36d88: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:00:51.524247 20886 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 312aa2c1b36b44998c5e3e4f77f36d88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6969cb95acbc49fa98783fb8853482c9" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 33049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:00:51.530757 20826 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9db07befbc1d45c0933bf9687bec1e5f [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:00:51.535255 20826 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9db07befbc1d45c0933bf9687bec1e5f [term 1 FOLLOWER]: Refusing update from remote peer 312aa2c1b36b44998c5e3e4f77f36d88: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:00:51.536408 20886 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 312aa2c1b36b44998c5e3e4f77f36d88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9db07befbc1d45c0933bf9687bec1e5f" member_type: VOTER last_known_addr { host: "127.29.112.188" port: 37973 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:00:51.557623 20891 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6969cb95acbc49fa98783fb8853482c9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 312aa2c1b36b44998c5e3e4f77f36d88. Latest consensus state: current_term: 1 leader_uuid: "312aa2c1b36b44998c5e3e4f77f36d88" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "312aa2c1b36b44998c5e3e4f77f36d88" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 45847 } } peers { permanent_uuid: "6969cb95acbc49fa98783fb8853482c9" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 33049 } } peers { permanent_uuid: "9db07befbc1d45c0933bf9687bec1e5f" member_type: VOTER last_known_addr { host: "127.29.112.188" port: 37973 } } }
I20260811 19:00:51.561784 20891 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6969cb95acbc49fa98783fb8853482c9 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 19:00:51.563691 20946 mvcc.cc:205] Tried to move back new op lower bound from 7317400991823650816 to 7317400991657799680. Current Snapshot: MvccSnapshot[applied={T|T < 7317400991823650816}]
I20260811 19:00:51.566164 20947 mvcc.cc:205] Tried to move back new op lower bound from 7317400991823650816 to 7317400991657799680. Current Snapshot: MvccSnapshot[applied={T|T < 7317400991823650816}]
I20260811 19:00:51.567538 20891 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6969cb95acbc49fa98783fb8853482c9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "312aa2c1b36b44998c5e3e4f77f36d88" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "312aa2c1b36b44998c5e3e4f77f36d88" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 45847 } } peers { permanent_uuid: "6969cb95acbc49fa98783fb8853482c9" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 33049 } } peers { permanent_uuid: "9db07befbc1d45c0933bf9687bec1e5f" member_type: VOTER last_known_addr { host: "127.29.112.188" port: 37973 } } }
I20260811 19:00:51.567870 20891 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6969cb95acbc49fa98783fb8853482c9 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 19:00:51.571532 20931 sys_catalog.cc:455] T 00000000000000000000000000000000 P 312aa2c1b36b44998c5e3e4f77f36d88 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "312aa2c1b36b44998c5e3e4f77f36d88" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "312aa2c1b36b44998c5e3e4f77f36d88" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 45847 } } peers { permanent_uuid: "6969cb95acbc49fa98783fb8853482c9" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 33049 } } peers { permanent_uuid: "9db07befbc1d45c0933bf9687bec1e5f" member_type: VOTER last_known_addr { host: "127.29.112.188" port: 37973 } } }
I20260811 19:00:51.572157 20931 sys_catalog.cc:458] T 00000000000000000000000000000000 P 312aa2c1b36b44998c5e3e4f77f36d88 [sys.catalog]: This master's current role is: LEADER
I20260811 19:00:51.577142 20935 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9db07befbc1d45c0933bf9687bec1e5f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 312aa2c1b36b44998c5e3e4f77f36d88. Latest consensus state: current_term: 1 leader_uuid: "312aa2c1b36b44998c5e3e4f77f36d88" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "312aa2c1b36b44998c5e3e4f77f36d88" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 45847 } } peers { permanent_uuid: "6969cb95acbc49fa98783fb8853482c9" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 33049 } } peers { permanent_uuid: "9db07befbc1d45c0933bf9687bec1e5f" member_type: VOTER last_known_addr { host: "127.29.112.188" port: 37973 } } }
I20260811 19:00:51.577577 20935 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9db07befbc1d45c0933bf9687bec1e5f [sys.catalog]: This master's current role is: FOLLOWER
I20260811 19:00:51.582484 20931 sys_catalog.cc:455] T 00000000000000000000000000000000 P 312aa2c1b36b44998c5e3e4f77f36d88 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "312aa2c1b36b44998c5e3e4f77f36d88" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "312aa2c1b36b44998c5e3e4f77f36d88" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 45847 } } peers { permanent_uuid: "6969cb95acbc49fa98783fb8853482c9" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 33049 } } peers { permanent_uuid: "9db07befbc1d45c0933bf9687bec1e5f" member_type: VOTER last_known_addr { host: "127.29.112.188" port: 37973 } } }
I20260811 19:00:51.583086 20938 catalog_manager.cc:1347] Generated new cluster ID: ebd465ca9dd54985982616f442fabc2b
I20260811 19:00:51.583209 20938 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:00:51.583081 20931 sys_catalog.cc:458] T 00000000000000000000000000000000 P 312aa2c1b36b44998c5e3e4f77f36d88 [sys.catalog]: This master's current role is: LEADER
I20260811 19:00:51.596305 20949 mvcc.cc:205] Tried to move back new op lower bound from 7317400991823650816 to 7317400991657799680. Current Snapshot: MvccSnapshot[applied={T|T < 7317400991823650816}]
I20260811 19:00:51.615726 20935 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9db07befbc1d45c0933bf9687bec1e5f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "312aa2c1b36b44998c5e3e4f77f36d88" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "312aa2c1b36b44998c5e3e4f77f36d88" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 45847 } } peers { permanent_uuid: "6969cb95acbc49fa98783fb8853482c9" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 33049 } } peers { permanent_uuid: "9db07befbc1d45c0933bf9687bec1e5f" member_type: VOTER last_known_addr { host: "127.29.112.188" port: 37973 } } }
I20260811 19:00:51.616073 20935 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9db07befbc1d45c0933bf9687bec1e5f [sys.catalog]: This master's current role is: FOLLOWER
I20260811 19:00:51.568238 20624 master_runner.cc:186] Error getting master registration for 127.29.112.188:37973: OK, instance_id { permanent_uuid: "9db07befbc1d45c0933bf9687bec1e5f" instance_seqno: 1786474851244692 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 19:00:51.618558 20624 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 19:00:51.628973 20938 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:00:51.631309 20938 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:00:51.647562 20970 catalog_manager.cc:1259] Loaded cluster ID: ebd465ca9dd54985982616f442fabc2b
I20260811 19:00:51.647758 20970 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9db07befbc1d45c0933bf9687bec1e5f: loading cluster ID for follower catalog manager: success
I20260811 19:00:51.648651 20938 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 312aa2c1b36b44998c5e3e4f77f36d88: Generated new TSK 0
I20260811 19:00:51.650210 20938 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 19:00:51.653008 20970 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9db07befbc1d45c0933bf9687bec1e5f: acquiring CA information for follower catalog manager: success
W20260811 19:00:51.655218 20970 catalog_manager.cc:873] Not found: no TSK found in the system table: failed to prepare follower catalog manager, will retry
I20260811 19:00:52.525924 20942 catalog_manager.cc:1259] Loaded cluster ID: ebd465ca9dd54985982616f442fabc2b
I20260811 19:00:52.526093 20942 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6969cb95acbc49fa98783fb8853482c9: loading cluster ID for follower catalog manager: success
I20260811 19:00:52.530690 20942 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6969cb95acbc49fa98783fb8853482c9: acquiring CA information for follower catalog manager: success
I20260811 19:00:52.533808 20942 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6969cb95acbc49fa98783fb8853482c9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 19:00:52.659204 20970 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9db07befbc1d45c0933bf9687bec1e5f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 19:00:52.781081 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 20497
I20260811 19:00:52.782730 20604 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:00:52.911175 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 20497
I20260811 19:00:52.960644 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 20624
I20260811 19:00:52.962106 20737 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:00:53.098198 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 20624
I20260811 19:00:53.137070 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 20760
I20260811 19:00:53.138394 20857 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:00:53.267439 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 20760
2026-08-11T19:00:53Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (6106 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474615205622-30146-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:00:53 dist-test-slave-h6n0 krb5kdc[20989](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:00:53 dist-test-slave-h6n0 krb5kdc[20989](info): set up 2 sockets
Aug 11 19:00:53 dist-test-slave-h6n0 krb5kdc[20989](info): commencing operation
krb5kdc: starting...
W20260811 19:00:54.789115 30146 mini_kdc.cc:121] Time spent starting KDC: real 1.468s user 0.003s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474615205622-30146-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474615205622-30146-0/krb5kdc/test-user.keytab.
Aug 11 19:00:55 dist-test-slave-h6n0 krb5kdc[20989](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474854, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:00:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:00:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.112.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab.
Entry for principal kudu/127.29.112.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab.
I20260811 19:00:55.138659 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474615205622-30146-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474615205622-30146-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474615205622-30146-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.190:34669
--webserver_interface=127.29.112.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab
--principal=kudu/127.29.112.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:37425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.190:34669
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:00:55.565349 21012 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:00:55.566990 21012 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:00:55.567162 21012 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:00:55.581290 21012 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:00:55.581878 21012 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:00:55.582018 21012 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:00:55.582435 21012 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:00:55.582660 21012 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:00:55.596199 21012 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:37425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474615205622-30146-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474615205622-30146-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.112.190:34669
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab
--principal=kudu/127.29.112.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.190:34669
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.112.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474615205622-30146-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:00:55.599783 21012 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:00:55.601497 21012 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:00:55.617193 21018 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:55.617252 21020 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:55.617274 21023 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:55.621543 21012 server_base.cc:1034] running on GCE node
I20260811 19:00:55.623035 21012 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:00:55.625129 21012 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:00:55.626468 21012 hybrid_clock.cc:648] HybridClock initialized: now 1786474855626392 us; error 65 us; skew 500 ppm
I20260811 19:00:55.626961 21012 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:00:55 dist-test-slave-h6n0 krb5kdc[20989](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474855, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:00:55.635399 21012 init.cc:377] Logged in from keytab as kudu/127.29.112.190@KRBTEST.COM (short username kudu)
I20260811 19:00:55.637480 21012 webserver.cc:460] Webserver started at http://127.29.112.190:45287/ using document root <none> and password file <none>
I20260811 19:00:55.638268 21012 fs_manager.cc:359] Metadata directory not provided
I20260811 19:00:55.638407 21012 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:00:55.638775 21012 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:00:55.642040 21012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474615205622-30146-0/minicluster-data/master-0/data/instance:
uuid: "aa5200503e65404d9bc23cd4241cf42b"
format_stamp: "Formatted at 2026-08-11 19:00:55 on dist-test-slave-h6n0"
server_key: "e60d2aa1d738e945b26eb9fd013e3325"
server_key_iv: "f3fb94ac6411019d37c561a95cbda4d8"
server_key_version: "encryptionkey@0"
I20260811 19:00:55.642868 21012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474615205622-30146-0/minicluster-data/master-0/wal/instance:
uuid: "aa5200503e65404d9bc23cd4241cf42b"
format_stamp: "Formatted at 2026-08-11 19:00:55 on dist-test-slave-h6n0"
server_key: "e60d2aa1d738e945b26eb9fd013e3325"
server_key_iv: "f3fb94ac6411019d37c561a95cbda4d8"
server_key_version: "encryptionkey@0"
I20260811 19:00:55.649144 21012 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 19:00:55.653702 21032 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:55.655506 21012 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260811 19:00:55.655748 21012 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474615205622-30146-0/minicluster-data/master-0/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474615205622-30146-0/minicluster-data/master-0/wal
uuid: "aa5200503e65404d9bc23cd4241cf42b"
format_stamp: "Formatted at 2026-08-11 19:00:55 on dist-test-slave-h6n0"
server_key: "e60d2aa1d738e945b26eb9fd013e3325"
server_key_iv: "f3fb94ac6411019d37c561a95cbda4d8"
server_key_version: "encryptionkey@0"
I20260811 19:00:55.655962 21012 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474615205622-30146-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474615205622-30146-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474615205622-30146-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:00:55.679801 21012 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:00:55.688493 21012 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:00:55.688831 21012 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:00:55.718513 21012 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.190:34669
I20260811 19:00:55.718715 21117 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.190:34669 every 8 connection(s)
I20260811 19:00:55.721711 21012 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
I20260811 19:00:55.728320 21119 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:00:55.729297 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 21012
I20260811 19:00:55.729566 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474615205622-30146-0/minicluster-data/master-0/wal/instance
I20260811 19:00:55.730290 30146 external_mini_cluster.cc:1428] Setting key cc27008bfd12c36f984493d72b14190f
I20260811 19:00:55.746424 21119 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa5200503e65404d9bc23cd4241cf42b: Bootstrap starting.
I20260811 19:00:55.751746 21119 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aa5200503e65404d9bc23cd4241cf42b: Neither blocks nor log segments found. Creating new log.
Aug 11 19:00:55 dist-test-slave-h6n0 krb5kdc[20989](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474854, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 19:00:55.753775 21119 log.cc:824] T 00000000000000000000000000000000 P aa5200503e65404d9bc23cd4241cf42b: Log is configured to *not* fsync() on all Append() calls
I20260811 19:00:55.758821 21119 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa5200503e65404d9bc23cd4241cf42b: No bootstrap required, opened a new log
I20260811 19:00:55.766260 21124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:55.733507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53880 (local address 127.29.112.190:34669)
0811 19:00:55.735173 (+ 1666us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:55.735199 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:55.735293 (+ 94us) server_negotiation.cc:407] Connection header received
0811 19:00:55.738176 (+ 2883us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:55.738212 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:55.738842 (+ 630us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:55.739360 (+ 518us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:55.741273 (+ 1913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:55.742887 (+ 1614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:55.744937 (+ 2050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:55.746380 (+ 1443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:55.746728 (+ 348us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:55.754282 (+ 7554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:55.754295 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:55.754319 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:55.754384 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:55.759021 (+ 4637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:55.760397 (+ 1376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:55.760409 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:55.760420 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:55.760847 (+ 427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:55.761541 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:55.761546 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:55.761548 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:55.762737 (+ 1189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:55.763172 (+ 435us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:55.763649 (+ 477us) server_negotiation.cc:299] Negotiation successful
0811 19:00:55.764459 (+ 810us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1263,"thread_start_us":604,"threads_started":1}
I20260811 19:00:55.772469 30146 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 19:00:55.773309 21119 raft_consensus.cc:348] T 00000000000000000000000000000000 P aa5200503e65404d9bc23cd4241cf42b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa5200503e65404d9bc23cd4241cf42b" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 34669 } }
I20260811 19:00:55.773813 21119 raft_consensus.cc:374] T 00000000000000000000000000000000 P aa5200503e65404d9bc23cd4241cf42b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:00:55.773978 21119 raft_consensus.cc:729] T 00000000000000000000000000000000 P aa5200503e65404d9bc23cd4241cf42b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa5200503e65404d9bc23cd4241cf42b, State: Initialized, Role: FOLLOWER
I20260811 19:00:55.774690 21119 consensus_queue.cc:260] T 00000000000000000000000000000000 P aa5200503e65404d9bc23cd4241cf42b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa5200503e65404d9bc23cd4241cf42b" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 34669 } }
I20260811 19:00:55.774989 21119 raft_consensus.cc:388] T 00000000000000000000000000000000 P aa5200503e65404d9bc23cd4241cf42b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:00:55.775154 21119 raft_consensus.cc:482] T 00000000000000000000000000000000 P aa5200503e65404d9bc23cd4241cf42b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:00:55.775362 21119 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aa5200503e65404d9bc23cd4241cf42b [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:00:55.777736 21119 raft_consensus.cc:504] T 00000000000000000000000000000000 P aa5200503e65404d9bc23cd4241cf42b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa5200503e65404d9bc23cd4241cf42b" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 34669 } }
I20260811 19:00:55.778314 21119 leader_election.cc:302] T 00000000000000000000000000000000 P aa5200503e65404d9bc23cd4241cf42b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aa5200503e65404d9bc23cd4241cf42b; no voters:
I20260811 19:00:55.778999 21119 leader_election.cc:288] T 00000000000000000000000000000000 P aa5200503e65404d9bc23cd4241cf42b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:00:55.779157 21126 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aa5200503e65404d9bc23cd4241cf42b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:00:55.780507 21126 raft_consensus.cc:686] T 00000000000000000000000000000000 P aa5200503e65404d9bc23cd4241cf42b [term 1 LEADER]: Becoming Leader. State: Replica: aa5200503e65404d9bc23cd4241cf42b, State: Running, Role: LEADER
I20260811 19:00:55.781126 21126 consensus_queue.cc:237] T 00000000000000000000000000000000 P aa5200503e65404d9bc23cd4241cf42b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa5200503e65404d9bc23cd4241cf42b" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 34669 } }
I20260811 19:00:55.781279 21119 sys_catalog.cc:564] T 00000000000000000000000000000000 P aa5200503e65404d9bc23cd4241cf42b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:00:55.784483 21127 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa5200503e65404d9bc23cd4241cf42b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aa5200503e65404d9bc23cd4241cf42b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa5200503e65404d9bc23cd4241cf42b" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 34669 } } }
I20260811 19:00:55.785017 21127 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa5200503e65404d9bc23cd4241cf42b [sys.catalog]: This master's current role is: LEADER
I20260811 19:00:55.784572 21128 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa5200503e65404d9bc23cd4241cf42b [sys.catalog]: SysCatalogTable state changed. Reason: New leader aa5200503e65404d9bc23cd4241cf42b. Latest consensus state: current_term: 1 leader_uuid: "aa5200503e65404d9bc23cd4241cf42b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa5200503e65404d9bc23cd4241cf42b" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 34669 } } }
I20260811 19:00:55.785935 21128 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa5200503e65404d9bc23cd4241cf42b [sys.catalog]: This master's current role is: LEADER
I20260811 19:00:55.787453 21136 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:00:55.792162 21136 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:00:55.803193 21136 catalog_manager.cc:1347] Generated new cluster ID: f971e3db50054907bdcaead0834af8a3
I20260811 19:00:55.803362 21136 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:00:55.821967 21136 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:00:55.823736 21136 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:00:55.831332 21136 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aa5200503e65404d9bc23cd4241cf42b: Generated new TSK 0
I20260811 19:00:55.833359 21136 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 19:00:55.889226 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 21012
I20260811 19:00:55.891196 21112 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:00:56.006589 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 21012
2026-08-11T19:00:56Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2746 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:00:56 dist-test-slave-h6n0 krb5kdc[21166](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:00:56 dist-test-slave-h6n0 krb5kdc[21166](info): set up 2 sockets
Aug 11 19:00:56 dist-test-slave-h6n0 krb5kdc[21166](info): commencing operation
krb5kdc: starting...
W20260811 19:00:57.666661 30146 mini_kdc.cc:121] Time spent starting KDC: real 1.598s user 0.005s sys 0.050s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/krb5kdc/test-user.keytab.
Aug 11 19:00:57 dist-test-slave-h6n0 krb5kdc[21166](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474857, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:00:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:00:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.112.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab.
Entry for principal kudu/127.29.112.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab.
I20260811 19:00:58.065158 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.190:37101
--webserver_interface=127.29.112.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab
--principal=kudu/127.29.112.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:41831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.190:37101
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:00:58.573712 21186 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:00:58.574090 21186 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:00:58.574334 21186 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:00:58.585579 21186 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:00:58.585757 21186 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:00:58.585875 21186 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:00:58.585983 21186 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:00:58.586071 21186 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:00:58.598769 21186 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:41831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.112.190:37101
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab
--principal=kudu/127.29.112.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.190:37101
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.112.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:00:58.600427 21186 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:00:58.602736 21186 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:00:58.617275 21195 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:58.617396 21199 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:58.617640 21204 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:58.618628 21186 server_base.cc:1034] running on GCE node
I20260811 19:00:58.619621 21186 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:00:58.621244 21186 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:00:58.622643 21186 hybrid_clock.cc:648] HybridClock initialized: now 1786474858622512 us; error 39 us; skew 500 ppm
I20260811 19:00:58.623049 21186 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:00:58 dist-test-slave-h6n0 krb5kdc[21166](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474858, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:00:58.628294 21186 init.cc:377] Logged in from keytab as kudu/127.29.112.190@KRBTEST.COM (short username kudu)
I20260811 19:00:58.630791 21186 webserver.cc:460] Webserver started at http://127.29.112.190:32779/ using document root <none> and password file <none>
I20260811 19:00:58.631635 21186 fs_manager.cc:359] Metadata directory not provided
I20260811 19:00:58.631803 21186 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:00:58.632153 21186 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:00:58.635668 21186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/master-0/data/instance:
uuid: "4985530a07eb4edeaa3cae40b71554d8"
format_stamp: "Formatted at 2026-08-11 19:00:58 on dist-test-slave-h6n0"
server_key: "664f1286906a295f39575e04fc41b0b6"
server_key_iv: "56d23316f6394ca240d593c3de306ea9"
server_key_version: "encryptionkey@0"
I20260811 19:00:58.636519 21186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/master-0/wal/instance:
uuid: "4985530a07eb4edeaa3cae40b71554d8"
format_stamp: "Formatted at 2026-08-11 19:00:58 on dist-test-slave-h6n0"
server_key: "664f1286906a295f39575e04fc41b0b6"
server_key_iv: "56d23316f6394ca240d593c3de306ea9"
server_key_version: "encryptionkey@0"
I20260811 19:00:58.642506 21186 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260811 19:00:58.647444 21217 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:58.649262 21186 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.004s
I20260811 19:00:58.649506 21186 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/master-0/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/master-0/wal
uuid: "4985530a07eb4edeaa3cae40b71554d8"
format_stamp: "Formatted at 2026-08-11 19:00:58 on dist-test-slave-h6n0"
server_key: "664f1286906a295f39575e04fc41b0b6"
server_key_iv: "56d23316f6394ca240d593c3de306ea9"
server_key_version: "encryptionkey@0"
I20260811 19:00:58.649782 21186 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:00:58.665194 21186 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:00:58.672504 21186 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:00:58.672904 21186 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:00:58.700168 21186 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.190:37101
I20260811 19:00:58.700191 21294 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.190:37101 every 8 connection(s)
I20260811 19:00:58.701749 21186 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
I20260811 19:00:58.706586 21295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:00:58.710471 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 21186
I20260811 19:00:58.710824 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/master-0/wal/instance
I20260811 19:00:58.711658 30146 external_mini_cluster.cc:1428] Setting key 4c6538acba400375137d742ed66b9a9c
I20260811 19:00:58.721465 21295 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4985530a07eb4edeaa3cae40b71554d8: Bootstrap starting.
I20260811 19:00:58.726245 21295 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4985530a07eb4edeaa3cae40b71554d8: Neither blocks nor log segments found. Creating new log.
I20260811 19:00:58.727852 21295 log.cc:824] T 00000000000000000000000000000000 P 4985530a07eb4edeaa3cae40b71554d8: Log is configured to *not* fsync() on all Append() calls
I20260811 19:00:58.734397 21295 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4985530a07eb4edeaa3cae40b71554d8: No bootstrap required, opened a new log
Aug 11 19:00:58 dist-test-slave-h6n0 krb5kdc[21166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474857, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 19:00:58.743975 21295 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4985530a07eb4edeaa3cae40b71554d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4985530a07eb4edeaa3cae40b71554d8" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 37101 } }
I20260811 19:00:58.744896 21295 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4985530a07eb4edeaa3cae40b71554d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:00:58.745381 21295 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4985530a07eb4edeaa3cae40b71554d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4985530a07eb4edeaa3cae40b71554d8, State: Initialized, Role: FOLLOWER
I20260811 19:00:58.747162 21295 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4985530a07eb4edeaa3cae40b71554d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4985530a07eb4edeaa3cae40b71554d8" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 37101 } }
I20260811 19:00:58.747969 21295 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4985530a07eb4edeaa3cae40b71554d8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:00:58.748402 21295 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4985530a07eb4edeaa3cae40b71554d8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:00:58.748844 21295 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4985530a07eb4edeaa3cae40b71554d8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:00:58.751458 21305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:58.714708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58898 (local address 127.29.112.190:37101)
0811 19:00:58.715672 (+ 964us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:58.715690 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:58.715748 (+ 58us) server_negotiation.cc:407] Connection header received
0811 19:00:58.719312 (+ 3564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:58.719352 (+ 40us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:58.719970 (+ 618us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:58.720502 (+ 532us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:58.723310 (+ 2808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:58.726022 (+ 2712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:58.728394 (+ 2372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:58.730091 (+ 1697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:58.730347 (+ 256us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:58.737736 (+ 7389us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:58.737746 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:58.737765 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:58.737816 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:58.743212 (+ 5396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:58.744851 (+ 1639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:58.744869 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:58.744890 (+ 21us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:58.745686 (+ 796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:58.746379 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:58.746384 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:58.746386 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:58.747435 (+ 1049us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:58.747856 (+ 421us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:58.748600 (+ 744us) server_negotiation.cc:299] Negotiation successful
0811 19:00:58.749358 (+ 758us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":679,"thread_start_us":414,"threads_started":1}
I20260811 19:00:58.755681 21295 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4985530a07eb4edeaa3cae40b71554d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4985530a07eb4edeaa3cae40b71554d8" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 37101 } }
I20260811 19:00:58.756812 21295 leader_election.cc:302] T 00000000000000000000000000000000 P 4985530a07eb4edeaa3cae40b71554d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4985530a07eb4edeaa3cae40b71554d8; no voters:
I20260811 19:00:58.757974 21295 leader_election.cc:288] T 00000000000000000000000000000000 P 4985530a07eb4edeaa3cae40b71554d8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:00:58.766680 21307 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4985530a07eb4edeaa3cae40b71554d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:00:58.770759 21295 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4985530a07eb4edeaa3cae40b71554d8 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:00:58.771754 21307 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4985530a07eb4edeaa3cae40b71554d8 [term 1 LEADER]: Becoming Leader. State: Replica: 4985530a07eb4edeaa3cae40b71554d8, State: Running, Role: LEADER
I20260811 19:00:58.773001 21307 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4985530a07eb4edeaa3cae40b71554d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4985530a07eb4edeaa3cae40b71554d8" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 37101 } }
I20260811 19:00:58.777787 21310 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4985530a07eb4edeaa3cae40b71554d8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4985530a07eb4edeaa3cae40b71554d8. Latest consensus state: current_term: 1 leader_uuid: "4985530a07eb4edeaa3cae40b71554d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4985530a07eb4edeaa3cae40b71554d8" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 37101 } } }
I20260811 19:00:58.777899 21308 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4985530a07eb4edeaa3cae40b71554d8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4985530a07eb4edeaa3cae40b71554d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4985530a07eb4edeaa3cae40b71554d8" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 37101 } } }
I20260811 19:00:58.778370 21310 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4985530a07eb4edeaa3cae40b71554d8 [sys.catalog]: This master's current role is: LEADER
I20260811 19:00:58.778487 21308 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4985530a07eb4edeaa3cae40b71554d8 [sys.catalog]: This master's current role is: LEADER
I20260811 19:00:58.781590 21323 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:00:58.790000 21323 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:00:58.801966 21323 catalog_manager.cc:1347] Generated new cluster ID: 3cfa2eb09efe418f8e8c0c1db4d80d41
I20260811 19:00:58.802383 21323 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:00:58.841921 21323 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:00:58.843641 21323 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:00:58.870985 21323 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4985530a07eb4edeaa3cae40b71554d8: Generated new TSK 0
I20260811 19:00:58.873485 21323 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.112.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.129.keytab.
Entry for principal kudu/127.29.112.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.129.keytab.
I20260811 19:00:58.971057 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.129:0
--local_ip_for_outbound_sockets=127.29.112.129
--webserver_interface=127.29.112.129
--webserver_port=0
--tserver_master_addrs=127.29.112.190:37101
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.129.keytab
--principal=kudu/127.29.112.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:41831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:00:59.405972 21341 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:00:59.406505 21341 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:00:59.406791 21341 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:00:59.415740 21341 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:00:59.415870 21341 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:00:59.416002 21341 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.129
I20260811 19:00:59.427928 21341 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:41831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.129.keytab
--principal=kudu/127.29.112.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.112.129
--webserver_port=0
--tserver_master_addrs=127.29.112.190:37101
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.129
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:00:59.429715 21341 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:00:59.431388 21341 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:00:59.448696 21350 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:59.448468 21352 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:00:59.448474 21355 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:00:59.449635 21341 server_base.cc:1034] running on GCE node
I20260811 19:00:59.450703 21341 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:00:59.452143 21341 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:00:59.453505 21341 hybrid_clock.cc:648] HybridClock initialized: now 1786474859453406 us; error 62 us; skew 500 ppm
I20260811 19:00:59.454162 21341 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:00:59 dist-test-slave-h6n0 krb5kdc[21166](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474859, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:00:59.458455 21341 init.cc:377] Logged in from keytab as kudu/127.29.112.129@KRBTEST.COM (short username kudu)
I20260811 19:00:59.460376 21341 webserver.cc:460] Webserver started at http://127.29.112.129:37943/ using document root <none> and password file <none>
I20260811 19:00:59.461151 21341 fs_manager.cc:359] Metadata directory not provided
I20260811 19:00:59.461266 21341 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:00:59.461597 21341 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:00:59.464792 21341 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-0/data/instance:
uuid: "c5540dcd5d1846028c6ee159697fc086"
format_stamp: "Formatted at 2026-08-11 19:00:59 on dist-test-slave-h6n0"
server_key: "e368153fff9f1dc9903d82a309663ea9"
server_key_iv: "0ff1c8300acc5725394f2b0c52a33c6c"
server_key_version: "encryptionkey@0"
I20260811 19:00:59.465629 21341 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-0/wal/instance:
uuid: "c5540dcd5d1846028c6ee159697fc086"
format_stamp: "Formatted at 2026-08-11 19:00:59 on dist-test-slave-h6n0"
server_key: "e368153fff9f1dc9903d82a309663ea9"
server_key_iv: "0ff1c8300acc5725394f2b0c52a33c6c"
server_key_version: "encryptionkey@0"
I20260811 19:00:59.471900 21341 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.003s
I20260811 19:00:59.477617 21364 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:59.479988 21341 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.001s
I20260811 19:00:59.480171 21341 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-0/wal
uuid: "c5540dcd5d1846028c6ee159697fc086"
format_stamp: "Formatted at 2026-08-11 19:00:59 on dist-test-slave-h6n0"
server_key: "e368153fff9f1dc9903d82a309663ea9"
server_key_iv: "0ff1c8300acc5725394f2b0c52a33c6c"
server_key_version: "encryptionkey@0"
I20260811 19:00:59.480453 21341 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:00:59.497991 21341 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:00:59.504225 21341 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:00:59.504608 21341 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:00:59.505738 21341 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:00:59.507385 21341 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:00:59.507472 21341 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:59.507707 21341 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:00:59.507865 21341 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:00:59.555557 21341 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.129:41521
I20260811 19:00:59.555632 21538 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.129:41521 every 8 connection(s)
I20260811 19:00:59.557021 21341 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-0/data/info.pb
I20260811 19:00:59.560590 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 21341
I20260811 19:00:59.561013 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-0/wal/instance
I20260811 19:00:59.561851 30146 external_mini_cluster.cc:1428] Setting key c9423f15d5b537e3ba17a889234c1483
Aug 11 19:00:59 dist-test-slave-h6n0 krb5kdc[21166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474859, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.129@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 19:00:59.588049 21542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:59.561079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.129:57409 (local address 127.29.112.190:37101)
0811 19:00:59.561607 (+ 528us) server_negotiation.cc:206] Beginning negotiation
0811 19:00:59.561614 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:00:59.563480 (+ 1866us) server_negotiation.cc:407] Connection header received
0811 19:00:59.566640 (+ 3160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:00:59.566652 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:00:59.566843 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:00:59.567002 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:00:59.569913 (+ 2911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:59.571198 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:59.574029 (+ 2831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:00:59.574763 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:00:59.575078 (+ 315us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:59.580320 (+ 5242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:00:59.580327 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:00:59.580330 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:00:59.580368 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:00:59.583449 (+ 3081us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:59.584403 (+ 954us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:59.584410 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:59.584416 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:59.584555 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:00:59.585341 (+ 786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:00:59.585350 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:00:59.585357 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:00:59.585931 (+ 574us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:00:59.586125 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0811 19:00:59.587191 (+ 1066us) server_negotiation.cc:299] Negotiation successful
0811 19:00:59.587497 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":396,"thread_start_us":263,"threads_started":1}
I20260811 19:00:59.588732 21543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:00:59.561847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37101 (local address 127.29.112.129:57409)
0811 19:00:59.563274 (+ 1427us) negotiation.cc:107] Waiting for socket to connect
0811 19:00:59.563340 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0811 19:00:59.565937 (+ 2597us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:00:59.567712 (+ 1775us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:00:59.567726 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:00:59.568845 (+ 1119us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:00:59.569586 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:59.569613 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:59.571547 (+ 1934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:59.571555 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:59.573822 (+ 2267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:00:59.573847 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:00:59.575366 (+ 1519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:00:59.575393 (+ 27us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:00:59.575736 (+ 343us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:00:59.577128 (+ 1392us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:00:59.577165 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:00:59.580026 (+ 2861us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:00:59.583634 (+ 3608us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:59.583645 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:59.583674 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:59.584264 (+ 590us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:59.584694 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:00:59.584700 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:00:59.584704 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:00:59.584965 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:00:59.586142 (+ 1177us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:00:59.586149 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:00:59.586740 (+ 591us) client_negotiation.cc:769] Sending connection context
0811 19:00:59.587017 (+ 277us) client_negotiation.cc:240] Negotiation successful
0811 19:00:59.587468 (+ 451us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1125,"thread_start_us":773,"threads_started":1}
I20260811 19:00:59.590516 21539 heartbeater.cc:347] Connected to a master server at 127.29.112.190:37101
I20260811 19:00:59.590919 21539 heartbeater.cc:464] Registering TS with master...
I20260811 19:00:59.591702 21539 heartbeater.cc:511] Master 127.29.112.190:37101 requested a full tablet report, sending...
I20260811 19:00:59.595772 21247 ts_manager.cc:194] Registered new tserver with Master: c5540dcd5d1846028c6ee159697fc086 (127.29.112.129:41521)
WARNING: no policy specified for kudu/127.29.112.130@KRBTEST.COM; defaulting to no policy
I20260811 19:00:59.597826 21247 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.129@KRBTEST.COM'} at 127.29.112.129:57409
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.130.keytab.
Entry for principal kudu/127.29.112.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.130.keytab.
I20260811 19:00:59.641578 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.130:0
--local_ip_for_outbound_sockets=127.29.112.130
--webserver_interface=127.29.112.130
--webserver_port=0
--tserver_master_addrs=127.29.112.190:37101
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.130.keytab
--principal=kudu/127.29.112.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:41831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:01:00.060094 21549 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:01:00.060567 21549 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:01:00.060688 21549 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:01:00.070706 21549 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:01:00.070948 21549 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:01:00.071403 21549 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.130
I20260811 19:01:00.084297 21549 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:41831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.130.keytab
--principal=kudu/127.29.112.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.112.130
--webserver_port=0
--tserver_master_addrs=127.29.112.190:37101
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.130
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:01:00.085914 21549 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:01:00.087427 21549 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:01:00.103296 21556 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:01:00.103230 21559 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:01:00.103557 21561 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:01:00.104174 21549 server_base.cc:1034] running on GCE node
I20260811 19:01:00.104884 21549 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:01:00.105919 21549 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:01:00.107198 21549 hybrid_clock.cc:648] HybridClock initialized: now 1786474860107128 us; error 54 us; skew 500 ppm
I20260811 19:01:00.107599 21549 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:01:00 dist-test-slave-h6n0 krb5kdc[21166](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474860, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:01:00.111799 21549 init.cc:377] Logged in from keytab as kudu/127.29.112.130@KRBTEST.COM (short username kudu)
I20260811 19:01:00.114171 21549 webserver.cc:460] Webserver started at http://127.29.112.130:40137/ using document root <none> and password file <none>
I20260811 19:01:00.115376 21549 fs_manager.cc:359] Metadata directory not provided
I20260811 19:01:00.115595 21549 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:01:00.116012 21549 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:01:00.119810 21549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-1/data/instance:
uuid: "ed49ffb15359470f84f5fae8705c97e4"
format_stamp: "Formatted at 2026-08-11 19:01:00 on dist-test-slave-h6n0"
server_key: "084c999a79f65c0a1b1a0cdf6a124b49"
server_key_iv: "7c1812b0d2ac818c81e65cb084c32743"
server_key_version: "encryptionkey@0"
I20260811 19:01:00.120718 21549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-1/wal/instance:
uuid: "ed49ffb15359470f84f5fae8705c97e4"
format_stamp: "Formatted at 2026-08-11 19:01:00 on dist-test-slave-h6n0"
server_key: "084c999a79f65c0a1b1a0cdf6a124b49"
server_key_iv: "7c1812b0d2ac818c81e65cb084c32743"
server_key_version: "encryptionkey@0"
I20260811 19:01:00.127434 21549 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260811 19:01:00.132282 21571 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:01:00.134263 21549 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260811 19:01:00.134665 21549 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-1/wal
uuid: "ed49ffb15359470f84f5fae8705c97e4"
format_stamp: "Formatted at 2026-08-11 19:01:00 on dist-test-slave-h6n0"
server_key: "084c999a79f65c0a1b1a0cdf6a124b49"
server_key_iv: "7c1812b0d2ac818c81e65cb084c32743"
server_key_version: "encryptionkey@0"
I20260811 19:01:00.134963 21549 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:01:00.158874 21549 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:01:00.164661 21549 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:01:00.165010 21549 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:01:00.166126 21549 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:01:00.168757 21549 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:01:00.169234 21549 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.001s user 0.000s sys 0.000s
I20260811 19:01:00.169598 21549 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:01:00.169678 21549 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:01:00.220517 21549 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.130:46063
I20260811 19:01:00.220768 21752 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.130:46063 every 8 connection(s)
I20260811 19:01:00.221904 21549 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-1/data/info.pb
I20260811 19:01:00.223237 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 21549
I20260811 19:01:00.223467 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-1/wal/instance
I20260811 19:01:00.224128 30146 external_mini_cluster.cc:1428] Setting key 2266b3b053dc7620313026f540386163
Aug 11 19:01:00 dist-test-slave-h6n0 krb5kdc[21166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474860, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.130@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 19:01:00.248790 21762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:00.225373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.130:53963 (local address 127.29.112.190:37101)
0811 19:01:00.225975 (+ 602us) server_negotiation.cc:206] Beginning negotiation
0811 19:01:00.225982 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:01:00.227399 (+ 1417us) server_negotiation.cc:407] Connection header received
0811 19:01:00.229243 (+ 1844us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:01:00.229249 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:01:00.229349 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:01:00.229496 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:01:00.231230 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:00.232041 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:00.233847 (+ 1806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:00.234346 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:00.234508 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:00.238989 (+ 4481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:01:00.238995 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:01:00.238998 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:01:00.239040 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:01:00.242439 (+ 3399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:01:00.244463 (+ 2024us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:01:00.244468 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:01:00.244471 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:01:00.244923 (+ 452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:01:00.245870 (+ 947us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:01:00.245875 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:01:00.245878 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:01:00.246510 (+ 632us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:01:00.246813 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0811 19:01:00.248096 (+ 1283us) server_negotiation.cc:299] Negotiation successful
0811 19:01:00.248341 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":429,"thread_start_us":285,"threads_started":1}
I20260811 19:01:00.250355 21764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:00.226007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37101 (local address 127.29.112.130:53963)
0811 19:01:00.226958 (+ 951us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:00.227010 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 19:01:00.228935 (+ 1925us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:01:00.229763 (+ 828us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:01:00.229772 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:01:00.230339 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:01:00.231045 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:01:00.231064 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:01:00.232227 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:01:00.232231 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:01:00.233675 (+ 1444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:01:00.233688 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:01:00.234655 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:01:00.234664 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:01:00.234914 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:00.235732 (+ 818us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:01:00.235763 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:01:00.238673 (+ 2910us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:01:00.242877 (+ 4204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:01:00.242896 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:01:00.242927 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:01:00.244230 (+ 1303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:01:00.245152 (+ 922us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:01:00.245160 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:01:00.245166 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:01:00.245645 (+ 479us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:01:00.246792 (+ 1147us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:01:00.246806 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:01:00.247454 (+ 648us) client_negotiation.cc:769] Sending connection context
0811 19:01:00.247914 (+ 460us) client_negotiation.cc:240] Negotiation successful
0811 19:01:00.248641 (+ 727us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":647,"thread_start_us":398,"threads_started":1}
I20260811 19:01:00.252830 21754 heartbeater.cc:347] Connected to a master server at 127.29.112.190:37101
I20260811 19:01:00.253274 21754 heartbeater.cc:464] Registering TS with master...
I20260811 19:01:00.254448 21754 heartbeater.cc:511] Master 127.29.112.190:37101 requested a full tablet report, sending...
I20260811 19:01:00.256839 21247 ts_manager.cc:194] Registered new tserver with Master: ed49ffb15359470f84f5fae8705c97e4 (127.29.112.130:46063)
I20260811 19:01:00.258638 21247 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.130@KRBTEST.COM'} at 127.29.112.130:53963
WARNING: no policy specified for kudu/127.29.112.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.131.keytab.
Entry for principal kudu/127.29.112.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.131.keytab.
I20260811 19:01:00.319916 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.131:0
--local_ip_for_outbound_sockets=127.29.112.131
--webserver_interface=127.29.112.131
--webserver_port=0
--tserver_master_addrs=127.29.112.190:37101
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.131.keytab
--principal=kudu/127.29.112.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:41831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 19:01:00.602310 21539 heartbeater.cc:503] Master 127.29.112.190:37101 was elected leader, sending a full tablet report...
W20260811 19:01:00.761758 21767 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:01:00.762245 21767 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:01:00.762442 21767 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:01:00.771003 21767 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:01:00.771140 21767 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:01:00.771288 21767 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.131
I20260811 19:01:00.782158 21767 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:41831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.131.keytab
--principal=kudu/127.29.112.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.112.131
--webserver_port=0
--tserver_master_addrs=127.29.112.190:37101
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.131
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:01:00.783595 21767 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:01:00.784915 21767 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:01:00.801105 21778 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:01:00.803664 21779 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:01:00.803949 21781 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:01:00.804839 21767 server_base.cc:1034] running on GCE node
I20260811 19:01:00.806078 21767 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:01:00.807683 21767 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:01:00.811120 21767 hybrid_clock.cc:648] HybridClock initialized: now 1786474860810936 us; error 177 us; skew 500 ppm
I20260811 19:01:00.811815 21767 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:01:00 dist-test-slave-h6n0 krb5kdc[21166](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474860, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:01:00.821671 21767 init.cc:377] Logged in from keytab as kudu/127.29.112.131@KRBTEST.COM (short username kudu)
I20260811 19:01:00.823798 21767 webserver.cc:460] Webserver started at http://127.29.112.131:38097/ using document root <none> and password file <none>
I20260811 19:01:00.824673 21767 fs_manager.cc:359] Metadata directory not provided
I20260811 19:01:00.824854 21767 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:01:00.825246 21767 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:01:00.829633 21767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-2/data/instance:
uuid: "8b70c8cd4d2e4edea782997b0b2663aa"
format_stamp: "Formatted at 2026-08-11 19:01:00 on dist-test-slave-h6n0"
server_key: "17151f16d18aa4af7c494b1266623e68"
server_key_iv: "e7d77d4a59c2b5dcd2cca775b8306f50"
server_key_version: "encryptionkey@0"
I20260811 19:01:00.831837 21767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-2/wal/instance:
uuid: "8b70c8cd4d2e4edea782997b0b2663aa"
format_stamp: "Formatted at 2026-08-11 19:01:00 on dist-test-slave-h6n0"
server_key: "17151f16d18aa4af7c494b1266623e68"
server_key_iv: "e7d77d4a59c2b5dcd2cca775b8306f50"
server_key_version: "encryptionkey@0"
I20260811 19:01:00.841710 21767 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.009s sys 0.000s
I20260811 19:01:00.847272 21792 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:01:00.849714 21767 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.000s sys 0.004s
I20260811 19:01:00.850057 21767 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-2/wal
uuid: "8b70c8cd4d2e4edea782997b0b2663aa"
format_stamp: "Formatted at 2026-08-11 19:01:00 on dist-test-slave-h6n0"
server_key: "17151f16d18aa4af7c494b1266623e68"
server_key_iv: "e7d77d4a59c2b5dcd2cca775b8306f50"
server_key_version: "encryptionkey@0"
I20260811 19:01:00.850898 21767 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:01:00.879499 21767 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:01:00.886579 21767 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:01:00.887069 21767 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:01:00.888309 21767 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:01:00.891433 21767 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:01:00.891675 21767 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:01:00.891912 21767 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:01:00.892131 21767 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:01:00.943356 21767 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.131:44959
I20260811 19:01:00.943661 21993 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.131:44959 every 8 connection(s)
I20260811 19:01:00.946121 21767 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-2/data/info.pb
I20260811 19:01:00.951823 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 21767
I20260811 19:01:00.957798 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-2/wal/instance
I20260811 19:01:00.958391 30146 external_mini_cluster.cc:1428] Setting key 3d3f353cfba08e85566361384c481442
Aug 11 19:01:00 dist-test-slave-h6n0 krb5kdc[21166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474860, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.131@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 19:01:00.973665 21998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:00.951021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.131:36895 (local address 127.29.112.190:37101)
0811 19:01:00.951914 (+ 893us) server_negotiation.cc:206] Beginning negotiation
0811 19:01:00.951927 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 19:01:00.953308 (+ 1381us) server_negotiation.cc:407] Connection header received
0811 19:01:00.955492 (+ 2184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:01:00.955501 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:01:00.955625 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:01:00.955820 (+ 195us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:01:00.958024 (+ 2204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:00.958907 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:00.960972 (+ 2065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:00.961650 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:00.961849 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:00.966095 (+ 4246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:01:00.966102 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:01:00.966106 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:01:00.966149 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:01:00.969474 (+ 3325us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:01:00.970391 (+ 917us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:01:00.970401 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:01:00.970408 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:01:00.970648 (+ 240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:01:00.971183 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:01:00.971193 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:01:00.971197 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:01:00.971581 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:01:00.971775 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0811 19:01:00.972825 (+ 1050us) server_negotiation.cc:299] Negotiation successful
0811 19:01:00.973137 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":640,"thread_start_us":409,"threads_started":1}
I20260811 19:01:00.975449 21999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:00.951839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37101 (local address 127.29.112.131:36895)
0811 19:01:00.953081 (+ 1242us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:00.953140 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 19:01:00.955179 (+ 2039us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:01:00.956385 (+ 1206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:01:00.956404 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:01:00.957143 (+ 739us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:01:00.957802 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:01:00.957831 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:01:00.959067 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:01:00.959080 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:01:00.960702 (+ 1622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:01:00.960717 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:01:00.961959 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:01:00.961971 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:01:00.962194 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:00.963082 (+ 888us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:01:00.963112 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:01:00.965807 (+ 2695us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:01:00.969665 (+ 3858us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:01:00.969677 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:01:00.969697 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:01:00.970222 (+ 525us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:01:00.970787 (+ 565us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:01:00.970794 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:01:00.970799 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:01:00.971037 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:01:00.971757 (+ 720us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:01:00.971767 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:01:00.972294 (+ 527us) client_negotiation.cc:769] Sending connection context
0811 19:01:00.972668 (+ 374us) client_negotiation.cc:240] Negotiation successful
0811 19:01:00.973332 (+ 664us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":877,"thread_start_us":605,"threads_started":1}
I20260811 19:01:00.977825 21994 heartbeater.cc:347] Connected to a master server at 127.29.112.190:37101
I20260811 19:01:00.978343 21994 heartbeater.cc:464] Registering TS with master...
I20260811 19:01:00.979470 21994 heartbeater.cc:511] Master 127.29.112.190:37101 requested a full tablet report, sending...
I20260811 19:01:00.981602 21247 ts_manager.cc:194] Registered new tserver with Master: 8b70c8cd4d2e4edea782997b0b2663aa (127.29.112.131:44959)
I20260811 19:01:00.983080 21247 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.131@KRBTEST.COM'} at 127.29.112.131:36895
I20260811 19:01:00.989655 30146 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 19:01:01.018278 21998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:00.998392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38346 (local address 127.29.112.190:37101)
0811 19:01:00.998831 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0811 19:01:00.998841 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 19:01:00.999291 (+ 450us) server_negotiation.cc:407] Connection header received
0811 19:01:00.999390 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:01:00.999394 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:01:00.999488 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:01:00.999584 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:01:01.002011 (+ 2427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:01.003146 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:01.005474 (+ 2328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:01.006054 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:01.006412 (+ 358us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:01.009941 (+ 3529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:01:01.009948 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:01:01.009951 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:01:01.010004 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:01:01.013505 (+ 3501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:01:01.014636 (+ 1131us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:01:01.014642 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:01:01.014645 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:01:01.014870 (+ 225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:01:01.015474 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:01:01.015480 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:01:01.015482 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:01:01.015999 (+ 517us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:01:01.016315 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0811 19:01:01.017167 (+ 852us) server_negotiation.cc:299] Negotiation successful
0811 19:01:01.017532 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":239}
I20260811 19:01:01.024724 21247 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38346:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 19:01:01.035003 21247 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 19:01:01.070905 22015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:01.053189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:41521 (local address 127.0.0.1:40100)
0811 19:01:01.055310 (+ 2121us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:01.055359 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 19:01:01.055595 (+ 236us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:01:01.057038 (+ 1443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:01:01.057050 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:01:01.057094 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:01:01.057684 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:01:01.057701 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:01:01.060486 (+ 2785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:01:01.060509 (+ 23us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:01:01.063765 (+ 3256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:01:01.063802 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:01:01.069258 (+ 5456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:01:01.069333 (+ 75us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:01:01.069737 (+ 404us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:01.069849 (+ 112us) client_negotiation.cc:769] Sending connection context
0811 19:01:01.070015 (+ 166us) client_negotiation.cc:240] Negotiation successful
0811 19:01:01.070329 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1788,"thread_start_us":1619,"threads_started":1}
I20260811 19:01:01.072254 22020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:01.054201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40100 (local address 127.29.112.129:41521)
0811 19:01:01.056300 (+ 2099us) server_negotiation.cc:206] Beginning negotiation
0811 19:01:01.056313 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 19:01:01.056353 (+ 40us) server_negotiation.cc:407] Connection header received
0811 19:01:01.056467 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:01:01.056476 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:01:01.056755 (+ 279us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:01:01.057912 (+ 1157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:01:01.058208 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:01.060270 (+ 2062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:01.064168 (+ 3898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:01.066969 (+ 2801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:01.071146 (+ 4177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:01.071516 (+ 370us) server_negotiation.cc:1035] Waiting for connection context
0811 19:01:01.071611 (+ 95us) server_negotiation.cc:299] Negotiation successful
0811 19:01:01.071866 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1899,"thread_start_us":1757,"threads_started":1}
I20260811 19:01:01.079522 22017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:01.053598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.131:44959 (local address 127.0.0.1:38886)
0811 19:01:01.067962 (+ 14364us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:01.067995 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0811 19:01:01.068242 (+ 247us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:01:01.069485 (+ 1243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:01:01.069496 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:01:01.069540 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:01:01.070110 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:01:01.070124 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:01:01.073055 (+ 2931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:01:01.073062 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:01:01.075572 (+ 2510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:01:01.075589 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:01:01.078071 (+ 2482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:01:01.078080 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:01:01.078610 (+ 530us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:01.078653 (+ 43us) client_negotiation.cc:769] Sending connection context
0811 19:01:01.078760 (+ 107us) client_negotiation.cc:240] Negotiation successful
0811 19:01:01.078966 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14127,"mutex_wait_us":92,"thread_start_us":13667,"threads_started":1}
I20260811 19:01:01.079743 22021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:01.054781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38886 (local address 127.29.112.131:44959)
0811 19:01:01.063366 (+ 8585us) server_negotiation.cc:206] Beginning negotiation
0811 19:01:01.063381 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 19:01:01.068268 (+ 4887us) server_negotiation.cc:407] Connection header received
0811 19:01:01.068493 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:01:01.068509 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:01:01.068991 (+ 482us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:01:01.070385 (+ 1394us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:01:01.070806 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:01.072845 (+ 2039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:01.075777 (+ 2932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:01.077729 (+ 1952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:01.078513 (+ 784us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:01.078815 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0811 19:01:01.078912 (+ 97us) server_negotiation.cc:299] Negotiation successful
0811 19:01:01.079145 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8312,"thread_start_us":8054,"threads_started":1}
I20260811 19:01:01.079109 21434 tablet_service.cc:1467] Processing CreateTablet for tablet bee57de780704e13b13e3b1c81284d80 (DEFAULT_TABLE table=test-table [id=a3f063c17fad4ba78167ccec6dfac869]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:01:01.081858 21434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bee57de780704e13b13e3b1c81284d80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:01:01.082681 22022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:01.056601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56176 (local address 127.29.112.130:46063)
0811 19:01:01.068938 (+ 12337us) server_negotiation.cc:206] Beginning negotiation
0811 19:01:01.068951 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 19:01:01.069896 (+ 945us) server_negotiation.cc:407] Connection header received
0811 19:01:01.070287 (+ 391us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:01:01.070298 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:01:01.070620 (+ 322us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:01:01.070878 (+ 258us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:01:01.072219 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:01.074837 (+ 2618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:01.080231 (+ 5394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:01.081457 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:01.081678 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:01.081892 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0811 19:01:01.082134 (+ 242us) server_negotiation.cc:299] Negotiation successful
0811 19:01:01.082307 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12167,"thread_start_us":12010,"threads_started":1}
I20260811 19:01:01.083356 22023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:01.056643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.130:46063 (local address 127.0.0.1:56176)
0811 19:01:01.069729 (+ 13086us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:01.069757 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0811 19:01:01.070040 (+ 283us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:01:01.071612 (+ 1572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:01:01.071616 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:01:01.071641 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:01:01.072027 (+ 386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:01:01.072036 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:01:01.075247 (+ 3211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:01:01.075280 (+ 33us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:01:01.079845 (+ 4565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:01:01.079879 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:01:01.081707 (+ 1828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:01:01.081715 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:01:01.081896 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:01.081943 (+ 47us) client_negotiation.cc:769] Sending connection context
0811 19:01:01.082072 (+ 129us) client_negotiation.cc:240] Negotiation successful
0811 19:01:01.082911 (+ 839us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12879,"mutex_wait_us":58,"thread_start_us":12579,"threads_started":1}
I20260811 19:01:01.085712 21879 tablet_service.cc:1467] Processing CreateTablet for tablet bee57de780704e13b13e3b1c81284d80 (DEFAULT_TABLE table=test-table [id=a3f063c17fad4ba78167ccec6dfac869]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:01:01.087585 21879 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bee57de780704e13b13e3b1c81284d80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:01:01.088593 21651 tablet_service.cc:1467] Processing CreateTablet for tablet bee57de780704e13b13e3b1c81284d80 (DEFAULT_TABLE table=test-table [id=a3f063c17fad4ba78167ccec6dfac869]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:01:01.090041 21651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bee57de780704e13b13e3b1c81284d80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:01:01.102329 22024 tablet_bootstrap.cc:492] T bee57de780704e13b13e3b1c81284d80 P c5540dcd5d1846028c6ee159697fc086: Bootstrap starting.
I20260811 19:01:01.105703 22025 tablet_bootstrap.cc:492] T bee57de780704e13b13e3b1c81284d80 P ed49ffb15359470f84f5fae8705c97e4: Bootstrap starting.
I20260811 19:01:01.107761 22026 tablet_bootstrap.cc:492] T bee57de780704e13b13e3b1c81284d80 P 8b70c8cd4d2e4edea782997b0b2663aa: Bootstrap starting.
I20260811 19:01:01.110095 22025 tablet_bootstrap.cc:654] T bee57de780704e13b13e3b1c81284d80 P ed49ffb15359470f84f5fae8705c97e4: Neither blocks nor log segments found. Creating new log.
I20260811 19:01:01.111063 22024 tablet_bootstrap.cc:654] T bee57de780704e13b13e3b1c81284d80 P c5540dcd5d1846028c6ee159697fc086: Neither blocks nor log segments found. Creating new log.
I20260811 19:01:01.111546 22025 log.cc:824] T bee57de780704e13b13e3b1c81284d80 P ed49ffb15359470f84f5fae8705c97e4: Log is configured to *not* fsync() on all Append() calls
I20260811 19:01:01.113511 22024 log.cc:824] T bee57de780704e13b13e3b1c81284d80 P c5540dcd5d1846028c6ee159697fc086: Log is configured to *not* fsync() on all Append() calls
I20260811 19:01:01.114322 22026 tablet_bootstrap.cc:654] T bee57de780704e13b13e3b1c81284d80 P 8b70c8cd4d2e4edea782997b0b2663aa: Neither blocks nor log segments found. Creating new log.
I20260811 19:01:01.116158 22026 log.cc:824] T bee57de780704e13b13e3b1c81284d80 P 8b70c8cd4d2e4edea782997b0b2663aa: Log is configured to *not* fsync() on all Append() calls
I20260811 19:01:01.116681 22025 tablet_bootstrap.cc:492] T bee57de780704e13b13e3b1c81284d80 P ed49ffb15359470f84f5fae8705c97e4: No bootstrap required, opened a new log
I20260811 19:01:01.117370 22025 ts_tablet_manager.cc:1397] T bee57de780704e13b13e3b1c81284d80 P ed49ffb15359470f84f5fae8705c97e4: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.010s
I20260811 19:01:01.119880 22024 tablet_bootstrap.cc:492] T bee57de780704e13b13e3b1c81284d80 P c5540dcd5d1846028c6ee159697fc086: No bootstrap required, opened a new log
I20260811 19:01:01.120445 22024 ts_tablet_manager.cc:1397] T bee57de780704e13b13e3b1c81284d80 P c5540dcd5d1846028c6ee159697fc086: Time spent bootstrapping tablet: real 0.018s user 0.007s sys 0.006s
I20260811 19:01:01.121368 22026 tablet_bootstrap.cc:492] T bee57de780704e13b13e3b1c81284d80 P 8b70c8cd4d2e4edea782997b0b2663aa: No bootstrap required, opened a new log
I20260811 19:01:01.121762 22026 ts_tablet_manager.cc:1397] T bee57de780704e13b13e3b1c81284d80 P 8b70c8cd4d2e4edea782997b0b2663aa: Time spent bootstrapping tablet: real 0.014s user 0.007s sys 0.004s
I20260811 19:01:01.138105 22024 raft_consensus.cc:348] T bee57de780704e13b13e3b1c81284d80 P c5540dcd5d1846028c6ee159697fc086 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5540dcd5d1846028c6ee159697fc086" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 41521 } } peers { permanent_uuid: "8b70c8cd4d2e4edea782997b0b2663aa" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 44959 } } peers { permanent_uuid: "ed49ffb15359470f84f5fae8705c97e4" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 46063 } }
I20260811 19:01:01.138741 22024 raft_consensus.cc:374] T bee57de780704e13b13e3b1c81284d80 P c5540dcd5d1846028c6ee159697fc086 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:01:01.138857 22024 raft_consensus.cc:729] T bee57de780704e13b13e3b1c81284d80 P c5540dcd5d1846028c6ee159697fc086 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5540dcd5d1846028c6ee159697fc086, State: Initialized, Role: FOLLOWER
I20260811 19:01:01.139259 22026 raft_consensus.cc:348] T bee57de780704e13b13e3b1c81284d80 P 8b70c8cd4d2e4edea782997b0b2663aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5540dcd5d1846028c6ee159697fc086" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 41521 } } peers { permanent_uuid: "8b70c8cd4d2e4edea782997b0b2663aa" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 44959 } } peers { permanent_uuid: "ed49ffb15359470f84f5fae8705c97e4" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 46063 } }
I20260811 19:01:01.139658 22026 raft_consensus.cc:374] T bee57de780704e13b13e3b1c81284d80 P 8b70c8cd4d2e4edea782997b0b2663aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:01:01.139793 22026 raft_consensus.cc:729] T bee57de780704e13b13e3b1c81284d80 P 8b70c8cd4d2e4edea782997b0b2663aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b70c8cd4d2e4edea782997b0b2663aa, State: Initialized, Role: FOLLOWER
I20260811 19:01:01.139734 22024 consensus_queue.cc:260] T bee57de780704e13b13e3b1c81284d80 P c5540dcd5d1846028c6ee159697fc086 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5540dcd5d1846028c6ee159697fc086" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 41521 } } peers { permanent_uuid: "8b70c8cd4d2e4edea782997b0b2663aa" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 44959 } } peers { permanent_uuid: "ed49ffb15359470f84f5fae8705c97e4" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 46063 } }
I20260811 19:01:01.140610 22026 consensus_queue.cc:260] T bee57de780704e13b13e3b1c81284d80 P 8b70c8cd4d2e4edea782997b0b2663aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5540dcd5d1846028c6ee159697fc086" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 41521 } } peers { permanent_uuid: "8b70c8cd4d2e4edea782997b0b2663aa" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 44959 } } peers { permanent_uuid: "ed49ffb15359470f84f5fae8705c97e4" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 46063 } }
I20260811 19:01:01.140924 22025 raft_consensus.cc:348] T bee57de780704e13b13e3b1c81284d80 P ed49ffb15359470f84f5fae8705c97e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5540dcd5d1846028c6ee159697fc086" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 41521 } } peers { permanent_uuid: "8b70c8cd4d2e4edea782997b0b2663aa" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 44959 } } peers { permanent_uuid: "ed49ffb15359470f84f5fae8705c97e4" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 46063 } }
I20260811 19:01:01.141444 22025 raft_consensus.cc:374] T bee57de780704e13b13e3b1c81284d80 P ed49ffb15359470f84f5fae8705c97e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:01:01.141568 22025 raft_consensus.cc:729] T bee57de780704e13b13e3b1c81284d80 P ed49ffb15359470f84f5fae8705c97e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed49ffb15359470f84f5fae8705c97e4, State: Initialized, Role: FOLLOWER
I20260811 19:01:01.142186 22025 consensus_queue.cc:260] T bee57de780704e13b13e3b1c81284d80 P ed49ffb15359470f84f5fae8705c97e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5540dcd5d1846028c6ee159697fc086" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 41521 } } peers { permanent_uuid: "8b70c8cd4d2e4edea782997b0b2663aa" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 44959 } } peers { permanent_uuid: "ed49ffb15359470f84f5fae8705c97e4" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 46063 } }
I20260811 19:01:01.143821 22024 ts_tablet_manager.cc:1428] T bee57de780704e13b13e3b1c81284d80 P c5540dcd5d1846028c6ee159697fc086: Time spent starting tablet: real 0.023s user 0.019s sys 0.001s
I20260811 19:01:01.146903 21754 heartbeater.cc:503] Master 127.29.112.190:37101 was elected leader, sending a full tablet report...
I20260811 19:01:01.146850 21994 heartbeater.cc:503] Master 127.29.112.190:37101 was elected leader, sending a full tablet report...
I20260811 19:01:01.149971 22026 ts_tablet_manager.cc:1428] T bee57de780704e13b13e3b1c81284d80 P 8b70c8cd4d2e4edea782997b0b2663aa: Time spent starting tablet: real 0.028s user 0.027s sys 0.000s
I20260811 19:01:01.151502 22025 ts_tablet_manager.cc:1428] T bee57de780704e13b13e3b1c81284d80 P ed49ffb15359470f84f5fae8705c97e4: Time spent starting tablet: real 0.034s user 0.027s sys 0.004s
I20260811 19:01:01.166813 22030 raft_consensus.cc:482] T bee57de780704e13b13e3b1c81284d80 P c5540dcd5d1846028c6ee159697fc086 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:01:01.167660 22030 raft_consensus.cc:504] T bee57de780704e13b13e3b1c81284d80 P c5540dcd5d1846028c6ee159697fc086 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5540dcd5d1846028c6ee159697fc086" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 41521 } } peers { permanent_uuid: "8b70c8cd4d2e4edea782997b0b2663aa" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 44959 } } peers { permanent_uuid: "ed49ffb15359470f84f5fae8705c97e4" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 46063 } }
I20260811 19:01:01.170130 22030 leader_election.cc:288] T bee57de780704e13b13e3b1c81284d80 P c5540dcd5d1846028c6ee159697fc086 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b70c8cd4d2e4edea782997b0b2663aa (127.29.112.131:44959), ed49ffb15359470f84f5fae8705c97e4 (127.29.112.130:46063)
I20260811 19:01:01.186393 22022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:01.171177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.129:52789 (local address 127.29.112.130:46063)
0811 19:01:01.171483 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0811 19:01:01.171495 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 19:01:01.174630 (+ 3135us) server_negotiation.cc:407] Connection header received
0811 19:01:01.174769 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:01:01.174776 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:01:01.174877 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:01:01.175001 (+ 124us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:01:01.176336 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:01.177864 (+ 1528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:01.182806 (+ 4942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:01.184490 (+ 1684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:01.184704 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:01.184789 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 19:01:01.185789 (+ 1000us) server_negotiation.cc:299] Negotiation successful
0811 19:01:01.186014 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260811 19:01:01.186371 22021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:01.172036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.129:40223 (local address 127.29.112.131:44959)
0811 19:01:01.172330 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0811 19:01:01.172341 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 19:01:01.174630 (+ 2289us) server_negotiation.cc:407] Connection header received
0811 19:01:01.174769 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:01:01.174776 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:01:01.174877 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:01:01.175009 (+ 132us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:01:01.176488 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:01.177861 (+ 1373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:01.182799 (+ 4938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:01.184490 (+ 1691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:01.184704 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:01.184789 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 19:01:01.185821 (+ 1032us) server_negotiation.cc:299] Negotiation successful
0811 19:01:01.186032 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 19:01:01.186425 22036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:01.171215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.131:44959 (local address 127.29.112.129:40223)
0811 19:01:01.174004 (+ 2789us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:01.174036 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0811 19:01:01.174328 (+ 292us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:01:01.175122 (+ 794us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:01:01.175129 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:01:01.175178 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:01:01.175973 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:01:01.175986 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:01:01.178288 (+ 2302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:01:01.178295 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:01:01.182412 (+ 4117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:01:01.182446 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:01:01.185028 (+ 2582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:01:01.185034 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:01:01.185408 (+ 374us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:01.185436 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 19:01:01.185566 (+ 130us) client_negotiation.cc:240] Negotiation successful
0811 19:01:01.185760 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2589,"mutex_wait_us":340,"thread_start_us":1340,"threads_started":1}
I20260811 19:01:01.186452 22037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:01.171215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.130:46063 (local address 127.29.112.129:52789)
0811 19:01:01.174219 (+ 3004us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:01.174239 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 19:01:01.174370 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:01:01.175122 (+ 752us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:01:01.175129 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:01:01.175188 (+ 59us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:01:01.175967 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:01:01.175986 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:01:01.178288 (+ 2302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:01:01.178296 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:01:01.182412 (+ 4116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:01:01.182446 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:01:01.184980 (+ 2534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:01:01.184990 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:01:01.185376 (+ 386us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:01.185427 (+ 51us) client_negotiation.cc:769] Sending connection context
0811 19:01:01.185566 (+ 139us) client_negotiation.cc:240] Negotiation successful
0811 19:01:01.185760 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2788,"mutex_wait_us":623,"thread_start_us":1325,"threads_started":1}
I20260811 19:01:01.188530 21916 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bee57de780704e13b13e3b1c81284d80" candidate_uuid: "c5540dcd5d1846028c6ee159697fc086" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b70c8cd4d2e4edea782997b0b2663aa" is_pre_election: true
I20260811 19:01:01.188711 21683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bee57de780704e13b13e3b1c81284d80" candidate_uuid: "c5540dcd5d1846028c6ee159697fc086" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed49ffb15359470f84f5fae8705c97e4" is_pre_election: true
I20260811 19:01:01.189512 21683 raft_consensus.cc:2454] T bee57de780704e13b13e3b1c81284d80 P ed49ffb15359470f84f5fae8705c97e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c5540dcd5d1846028c6ee159697fc086 in term 0.
I20260811 19:01:01.189610 21916 raft_consensus.cc:2454] T bee57de780704e13b13e3b1c81284d80 P 8b70c8cd4d2e4edea782997b0b2663aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c5540dcd5d1846028c6ee159697fc086 in term 0.
I20260811 19:01:01.190629 21367 leader_election.cc:302] T bee57de780704e13b13e3b1c81284d80 P c5540dcd5d1846028c6ee159697fc086 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b70c8cd4d2e4edea782997b0b2663aa, c5540dcd5d1846028c6ee159697fc086; no voters:
I20260811 19:01:01.191329 22030 raft_consensus.cc:2781] T bee57de780704e13b13e3b1c81284d80 P c5540dcd5d1846028c6ee159697fc086 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 19:01:01.192058 22030 raft_consensus.cc:482] T bee57de780704e13b13e3b1c81284d80 P c5540dcd5d1846028c6ee159697fc086 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 19:01:01.192418 22030 raft_consensus.cc:3037] T bee57de780704e13b13e3b1c81284d80 P c5540dcd5d1846028c6ee159697fc086 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:01:01.196005 22030 raft_consensus.cc:504] T bee57de780704e13b13e3b1c81284d80 P c5540dcd5d1846028c6ee159697fc086 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5540dcd5d1846028c6ee159697fc086" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 41521 } } peers { permanent_uuid: "8b70c8cd4d2e4edea782997b0b2663aa" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 44959 } } peers { permanent_uuid: "ed49ffb15359470f84f5fae8705c97e4" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 46063 } }
I20260811 19:01:01.197216 22030 leader_election.cc:288] T bee57de780704e13b13e3b1c81284d80 P c5540dcd5d1846028c6ee159697fc086 [CANDIDATE]: Term 1 election: Requested vote from peers 8b70c8cd4d2e4edea782997b0b2663aa (127.29.112.131:44959), ed49ffb15359470f84f5fae8705c97e4 (127.29.112.130:46063)
I20260811 19:01:01.198194 21916 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bee57de780704e13b13e3b1c81284d80" candidate_uuid: "c5540dcd5d1846028c6ee159697fc086" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b70c8cd4d2e4edea782997b0b2663aa"
I20260811 19:01:01.198194 21683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bee57de780704e13b13e3b1c81284d80" candidate_uuid: "c5540dcd5d1846028c6ee159697fc086" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed49ffb15359470f84f5fae8705c97e4"
I20260811 19:01:01.198486 21916 raft_consensus.cc:3037] T bee57de780704e13b13e3b1c81284d80 P 8b70c8cd4d2e4edea782997b0b2663aa [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:01:01.198475 21683 raft_consensus.cc:3037] T bee57de780704e13b13e3b1c81284d80 P ed49ffb15359470f84f5fae8705c97e4 [term 0 FOLLOWER]: Advancing to term 1
W20260811 19:01:01.199208 21996 tablet.cc:2369] T bee57de780704e13b13e3b1c81284d80 P 8b70c8cd4d2e4edea782997b0b2663aa: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 19:01:01.202144 21916 raft_consensus.cc:2454] T bee57de780704e13b13e3b1c81284d80 P 8b70c8cd4d2e4edea782997b0b2663aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c5540dcd5d1846028c6ee159697fc086 in term 1.
I20260811 19:01:01.202144 21683 raft_consensus.cc:2454] T bee57de780704e13b13e3b1c81284d80 P ed49ffb15359470f84f5fae8705c97e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c5540dcd5d1846028c6ee159697fc086 in term 1.
I20260811 19:01:01.202893 21367 leader_election.cc:302] T bee57de780704e13b13e3b1c81284d80 P c5540dcd5d1846028c6ee159697fc086 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 8b70c8cd4d2e4edea782997b0b2663aa, c5540dcd5d1846028c6ee159697fc086, ed49ffb15359470f84f5fae8705c97e4; no voters:
I20260811 19:01:01.203416 22030 raft_consensus.cc:2781] T bee57de780704e13b13e3b1c81284d80 P c5540dcd5d1846028c6ee159697fc086 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:01:01.204077 22030 raft_consensus.cc:686] T bee57de780704e13b13e3b1c81284d80 P c5540dcd5d1846028c6ee159697fc086 [term 1 LEADER]: Becoming Leader. State: Replica: c5540dcd5d1846028c6ee159697fc086, State: Running, Role: LEADER
I20260811 19:01:01.204924 22030 consensus_queue.cc:237] T bee57de780704e13b13e3b1c81284d80 P c5540dcd5d1846028c6ee159697fc086 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5540dcd5d1846028c6ee159697fc086" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 41521 } } peers { permanent_uuid: "8b70c8cd4d2e4edea782997b0b2663aa" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 44959 } } peers { permanent_uuid: "ed49ffb15359470f84f5fae8705c97e4" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 46063 } }
I20260811 19:01:01.212585 21245 catalog_manager.cc:5495] T bee57de780704e13b13e3b1c81284d80 P c5540dcd5d1846028c6ee159697fc086 reported cstate change: term changed from 0 to 1, leader changed from <none> to c5540dcd5d1846028c6ee159697fc086 (127.29.112.129). New cstate: current_term: 1 leader_uuid: "c5540dcd5d1846028c6ee159697fc086" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5540dcd5d1846028c6ee159697fc086" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 41521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b70c8cd4d2e4edea782997b0b2663aa" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 44959 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed49ffb15359470f84f5fae8705c97e4" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 46063 } health_report { overall_health: UNKNOWN } } }
W20260811 19:01:01.225391 21755 tablet.cc:2369] T bee57de780704e13b13e3b1c81284d80 P ed49ffb15359470f84f5fae8705c97e4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 19:01:01.260229 22020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:01.250486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40104 (local address 127.29.112.129:41521)
0811 19:01:01.250769 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0811 19:01:01.250774 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:01:01.250798 (+ 24us) server_negotiation.cc:407] Connection header received
0811 19:01:01.251507 (+ 709us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:01:01.251512 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:01:01.251647 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:01:01.251836 (+ 189us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:01:01.252854 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:01.254228 (+ 1374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:01.256647 (+ 2419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:01.257292 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:01.257439 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:01.258347 (+ 908us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:01:01.258955 (+ 608us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:01:01.259126 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0811 19:01:01.259631 (+ 505us) server_negotiation.cc:299] Negotiation successful
0811 19:01:01.259745 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133}
I20260811 19:01:01.271553 21916 raft_consensus.cc:1261] T bee57de780704e13b13e3b1c81284d80 P 8b70c8cd4d2e4edea782997b0b2663aa [term 1 FOLLOWER]: Refusing update from remote peer c5540dcd5d1846028c6ee159697fc086: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:01:01.271589 21683 raft_consensus.cc:1261] T bee57de780704e13b13e3b1c81284d80 P ed49ffb15359470f84f5fae8705c97e4 [term 1 FOLLOWER]: Refusing update from remote peer c5540dcd5d1846028c6ee159697fc086: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:01:01.273772 22038 consensus_queue.cc:1035] T bee57de780704e13b13e3b1c81284d80 P c5540dcd5d1846028c6ee159697fc086 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed49ffb15359470f84f5fae8705c97e4" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 46063 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:01:01.274478 22030 consensus_queue.cc:1035] T bee57de780704e13b13e3b1c81284d80 P c5540dcd5d1846028c6ee159697fc086 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b70c8cd4d2e4edea782997b0b2663aa" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 44959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:01:01.302312 22048 mvcc.cc:205] Tried to move back new op lower bound from 7317401031754371072 to 7317401031503712256. Current Snapshot: MvccSnapshot[applied={T|T < 7317401031754371072}]
I20260811 19:01:01.317935 21246 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38346:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:01:01.318521 21246 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38346:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:01:01.324949 21246 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4985530a07eb4edeaa3cae40b71554d8: Sending DeleteTablet for 3 replicas of tablet bee57de780704e13b13e3b1c81284d80
I20260811 19:01:01.328794 21879 tablet_service.cc:1514] Processing DeleteTablet for tablet bee57de780704e13b13e3b1c81284d80 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:01:01 UTC) from {username='kudu', principal='kudu/127.29.112.190@KRBTEST.COM'} at 127.0.0.1:38886
I20260811 19:01:01.334911 21434 tablet_service.cc:1514] Processing DeleteTablet for tablet bee57de780704e13b13e3b1c81284d80 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:01:01 UTC) from {username='kudu', principal='kudu/127.29.112.190@KRBTEST.COM'} at 127.0.0.1:40100
I20260811 19:01:01.335776 21651 tablet_service.cc:1514] Processing DeleteTablet for tablet bee57de780704e13b13e3b1c81284d80 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:01:01 UTC) from {username='kudu', principal='kudu/127.29.112.190@KRBTEST.COM'} at 127.0.0.1:56176
I20260811 19:01:01.341811 22071 tablet_replica.cc:330] T bee57de780704e13b13e3b1c81284d80 P c5540dcd5d1846028c6ee159697fc086: stopping tablet replica
I20260811 19:01:01.342495 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 21341
I20260811 19:01:01.342696 22071 raft_consensus.cc:2229] T bee57de780704e13b13e3b1c81284d80 P c5540dcd5d1846028c6ee159697fc086 [term 1 LEADER]: Raft consensus shutting down.
I20260811 19:01:01.343530 22071 raft_consensus.cc:2258] T bee57de780704e13b13e3b1c81284d80 P c5540dcd5d1846028c6ee159697fc086 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:01:01.344972 22069 tablet_replica.cc:330] T bee57de780704e13b13e3b1c81284d80 P 8b70c8cd4d2e4edea782997b0b2663aa: stopping tablet replica
I20260811 19:01:01.348462 22069 raft_consensus.cc:2229] T bee57de780704e13b13e3b1c81284d80 P 8b70c8cd4d2e4edea782997b0b2663aa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:01:01.350143 22069 raft_consensus.cc:2258] T bee57de780704e13b13e3b1c81284d80 P 8b70c8cd4d2e4edea782997b0b2663aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:01:01.353852 22075 tablet_replica.cc:330] T bee57de780704e13b13e3b1c81284d80 P ed49ffb15359470f84f5fae8705c97e4: stopping tablet replica
I20260811 19:01:01.354707 22075 raft_consensus.cc:2229] T bee57de780704e13b13e3b1c81284d80 P ed49ffb15359470f84f5fae8705c97e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:01:01.356539 22075 raft_consensus.cc:2258] T bee57de780704e13b13e3b1c81284d80 P ed49ffb15359470f84f5fae8705c97e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:01:01.357791 22069 ts_tablet_manager.cc:1905] T bee57de780704e13b13e3b1c81284d80 P 8b70c8cd4d2e4edea782997b0b2663aa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:01:01.352456 22071 ts_tablet_manager.cc:1905] T bee57de780704e13b13e3b1c81284d80 P c5540dcd5d1846028c6ee159697fc086: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:01:01.369210 22069 ts_tablet_manager.cc:1918] T bee57de780704e13b13e3b1c81284d80 P 8b70c8cd4d2e4edea782997b0b2663aa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:01:01.369601 22069 log.cc:1196] T bee57de780704e13b13e3b1c81284d80 P 8b70c8cd4d2e4edea782997b0b2663aa: Deleting WAL directory at /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-2/wal/wals/bee57de780704e13b13e3b1c81284d80
I20260811 19:01:01.370319 22069 ts_tablet_manager.cc:1939] T bee57de780704e13b13e3b1c81284d80 P 8b70c8cd4d2e4edea782997b0b2663aa: Deleting consensus metadata
I20260811 19:01:01.372710 22071 ts_tablet_manager.cc:1918] T bee57de780704e13b13e3b1c81284d80 P c5540dcd5d1846028c6ee159697fc086: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:01:01.373153 22071 log.cc:1196] T bee57de780704e13b13e3b1c81284d80 P c5540dcd5d1846028c6ee159697fc086: Deleting WAL directory at /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-0/wal/wals/bee57de780704e13b13e3b1c81284d80
I20260811 19:01:01.373919 21220 catalog_manager.cc:4841] TS 8b70c8cd4d2e4edea782997b0b2663aa (127.29.112.131:44959): tablet bee57de780704e13b13e3b1c81284d80 (table test-table [id=a3f063c17fad4ba78167ccec6dfac869]) successfully deleted
I20260811 19:01:01.374646 22071 ts_tablet_manager.cc:1939] T bee57de780704e13b13e3b1c81284d80 P c5540dcd5d1846028c6ee159697fc086: Deleting consensus metadata
I20260811 19:01:01.375512 22075 ts_tablet_manager.cc:1905] T bee57de780704e13b13e3b1c81284d80 P ed49ffb15359470f84f5fae8705c97e4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:01:01.377665 21219 catalog_manager.cc:4841] TS c5540dcd5d1846028c6ee159697fc086 (127.29.112.129:41521): tablet bee57de780704e13b13e3b1c81284d80 (table test-table [id=a3f063c17fad4ba78167ccec6dfac869]) successfully deleted
Aug 11 19:01:01 dist-test-slave-h6n0 krb5kdc[21166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474857, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.129@KRBTEST.COM
I20260811 19:01:01.384042 22075 ts_tablet_manager.cc:1918] T bee57de780704e13b13e3b1c81284d80 P ed49ffb15359470f84f5fae8705c97e4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:01:01.384254 22075 log.cc:1196] T bee57de780704e13b13e3b1c81284d80 P ed49ffb15359470f84f5fae8705c97e4: Deleting WAL directory at /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-1/wal/wals/bee57de780704e13b13e3b1c81284d80
I20260811 19:01:01.384927 22075 ts_tablet_manager.cc:1939] T bee57de780704e13b13e3b1c81284d80 P ed49ffb15359470f84f5fae8705c97e4: Deleting consensus metadata
I20260811 19:01:01.387838 21219 catalog_manager.cc:4841] TS ed49ffb15359470f84f5fae8705c97e4 (127.29.112.130:46063): tablet bee57de780704e13b13e3b1c81284d80 (table test-table [id=a3f063c17fad4ba78167ccec6dfac869]) successfully deleted
I20260811 19:01:01.394213 22020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:01.344003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40118 (local address 127.29.112.129:41521)
0811 19:01:01.344568 (+ 565us) server_negotiation.cc:206] Beginning negotiation
0811 19:01:01.344585 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 19:01:01.353652 (+ 9067us) server_negotiation.cc:407] Connection header received
0811 19:01:01.354398 (+ 746us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:01:01.354405 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:01:01.354558 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:01:01.355652 (+ 1094us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:01:01.360936 (+ 5284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:01.362868 (+ 1932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:01.366080 (+ 3212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:01.366954 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:01.367073 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:01.383441 (+ 16368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:01:01.383475 (+ 34us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:01:01.383500 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:01:01.383573 (+ 73us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:01:01.389107 (+ 5534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:01:01.390221 (+ 1114us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:01:01.390232 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:01:01.390237 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:01:01.390468 (+ 231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:01:01.391459 (+ 991us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:01:01.391466 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:01:01.391470 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:01:01.392083 (+ 613us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:01:01.392469 (+ 386us) server_negotiation.cc:1035] Waiting for connection context
0811 19:01:01.393297 (+ 828us) server_negotiation.cc:299] Negotiation successful
0811 19:01:01.393620 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 19:01:01.395511 21534 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:01:01.535009 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 21341
I20260811 19:01:01.578172 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 21549
Aug 11 19:01:01 dist-test-slave-h6n0 krb5kdc[21166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474857, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.130@KRBTEST.COM
I20260811 19:01:01.598930 22022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:01.579311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56182 (local address 127.29.112.130:46063)
0811 19:01:01.579568 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0811 19:01:01.579576 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:01:01.579683 (+ 107us) server_negotiation.cc:407] Connection header received
0811 19:01:01.579904 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:01:01.579910 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:01:01.579993 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:01:01.580107 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:01:01.581604 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:01.582366 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:01.583664 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:01.584083 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:01.584508 (+ 425us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:01.591143 (+ 6635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:01:01.591165 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:01:01.591183 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:01:01.591237 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:01:01.595656 (+ 4419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:01:01.596601 (+ 945us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:01:01.596609 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:01:01.596612 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:01:01.596769 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:01:01.597167 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:01:01.597172 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:01:01.597175 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:01:01.597615 (+ 440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:01:01.597950 (+ 335us) server_negotiation.cc:1035] Waiting for connection context
0811 19:01:01.598245 (+ 295us) server_negotiation.cc:299] Negotiation successful
0811 19:01:01.598502 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260811 19:01:01.599722 21746 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:01:01.748106 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 21549
I20260811 19:01:01.786263 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 21767
Aug 11 19:01:01 dist-test-slave-h6n0 krb5kdc[21166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474857, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.131@KRBTEST.COM
I20260811 19:01:01.815492 22080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:01.787784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38902 (local address 127.29.112.131:44959)
0811 19:01:01.788416 (+ 632us) server_negotiation.cc:206] Beginning negotiation
0811 19:01:01.788421 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:01:01.788443 (+ 22us) server_negotiation.cc:407] Connection header received
0811 19:01:01.788521 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:01:01.788525 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:01:01.788621 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:01:01.788751 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:01:01.794269 (+ 5518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:01.795414 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:01.798375 (+ 2961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:01.799021 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:01.799213 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:01.805842 (+ 6629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:01:01.805865 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:01:01.805886 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:01:01.805944 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:01:01.811369 (+ 5425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:01:01.812413 (+ 1044us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:01:01.812420 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:01:01.812423 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:01:01.812590 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:01:01.813254 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:01:01.813261 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:01:01.813265 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:01:01.813696 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:01:01.814008 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0811 19:01:01.814749 (+ 741us) server_negotiation.cc:299] Negotiation successful
0811 19:01:01.815039 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":518,"thread_start_us":371,"threads_started":1}
I20260811 19:01:01.816181 21985 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:01:01.943779 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 21767
I20260811 19:01:01.983965 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 21186
I20260811 19:01:01.985800 21288 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:01:02.146862 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 21186
I20260811 19:01:02.195376 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.190:37101
--webserver_interface=127.29.112.190
--webserver_port=32779
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab
--principal=kudu/127.29.112.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:41831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.190:37101
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:01:02.712478 22084 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:01:02.713030 22084 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:01:02.713399 22084 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:01:02.731364 22084 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:01:02.731608 22084 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:01:02.731741 22084 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:01:02.731868 22084 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:01:02.731957 22084 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:01:02.748801 22084 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:41831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.29.112.190:37101
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab
--principal=kudu/127.29.112.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.190:37101
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.112.190
--webserver_port=32779
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:01:02.750916 22084 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:01:02.752583 22084 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:01:02.783347 22097 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:01:02.783439 22098 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:01:02.783411 22100 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:01:02.786787 22084 server_base.cc:1034] running on GCE node
I20260811 19:01:02.789547 22084 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:01:02.793653 22084 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:01:02.795603 22084 hybrid_clock.cc:648] HybridClock initialized: now 1786474862795602 us; error 253 us; skew 500 ppm
I20260811 19:01:02.796481 22084 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:01:02 dist-test-slave-h6n0 krb5kdc[21166](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474862, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:01:02.803376 22084 init.cc:377] Logged in from keytab as kudu/127.29.112.190@KRBTEST.COM (short username kudu)
I20260811 19:01:02.807847 22084 webserver.cc:460] Webserver started at http://127.29.112.190:32779/ using document root <none> and password file <none>
I20260811 19:01:02.810329 22084 fs_manager.cc:359] Metadata directory not provided
I20260811 19:01:02.810738 22084 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:01:02.820508 22084 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 19:01:02.824158 22113 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:01:02.826361 22084 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 19:01:02.826637 22084 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/master-0/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/master-0/wal
uuid: "4985530a07eb4edeaa3cae40b71554d8"
format_stamp: "Formatted at 2026-08-11 19:00:58 on dist-test-slave-h6n0"
server_key: "664f1286906a295f39575e04fc41b0b6"
server_key_iv: "56d23316f6394ca240d593c3de306ea9"
server_key_version: "encryptionkey@0"
I20260811 19:01:02.827977 22084 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:01:02.854071 22084 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:01:02.859987 22084 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:01:02.860365 22084 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:01:02.888008 22084 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.190:37101
I20260811 19:01:02.888036 22198 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.190:37101 every 8 connection(s)
I20260811 19:01:02.890580 22084 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
I20260811 19:01:02.897759 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 22084
I20260811 19:01:02.899099 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.129:41521
--local_ip_for_outbound_sockets=127.29.112.129
--tserver_master_addrs=127.29.112.190:37101
--webserver_port=37943
--webserver_interface=127.29.112.129
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.129.keytab
--principal=kudu/127.29.112.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:41831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 19:01:02.900938 22201 sys_catalog.cc:263] Verifying existing consensus state
I20260811 19:01:02.908399 22201 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4985530a07eb4edeaa3cae40b71554d8: Bootstrap starting.
I20260811 19:01:02.938082 22201 log.cc:824] T 00000000000000000000000000000000 P 4985530a07eb4edeaa3cae40b71554d8: Log is configured to *not* fsync() on all Append() calls
I20260811 19:01:02.954969 22201 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4985530a07eb4edeaa3cae40b71554d8: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 19:01:02.955886 22201 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4985530a07eb4edeaa3cae40b71554d8: Bootstrap complete.
I20260811 19:01:02.968482 22201 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4985530a07eb4edeaa3cae40b71554d8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4985530a07eb4edeaa3cae40b71554d8" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 37101 } }
I20260811 19:01:02.970063 22201 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4985530a07eb4edeaa3cae40b71554d8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4985530a07eb4edeaa3cae40b71554d8, State: Initialized, Role: FOLLOWER
I20260811 19:01:02.971114 22201 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4985530a07eb4edeaa3cae40b71554d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4985530a07eb4edeaa3cae40b71554d8" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 37101 } }
I20260811 19:01:02.972002 22201 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4985530a07eb4edeaa3cae40b71554d8 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:01:02.972317 22201 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4985530a07eb4edeaa3cae40b71554d8 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:01:02.972782 22201 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4985530a07eb4edeaa3cae40b71554d8 [term 1 FOLLOWER]: Advancing to term 2
I20260811 19:01:02.978726 22201 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4985530a07eb4edeaa3cae40b71554d8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4985530a07eb4edeaa3cae40b71554d8" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 37101 } }
I20260811 19:01:02.979434 22201 leader_election.cc:302] T 00000000000000000000000000000000 P 4985530a07eb4edeaa3cae40b71554d8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4985530a07eb4edeaa3cae40b71554d8; no voters:
I20260811 19:01:02.980163 22201 leader_election.cc:288] T 00000000000000000000000000000000 P 4985530a07eb4edeaa3cae40b71554d8 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 19:01:02.980325 22212 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4985530a07eb4edeaa3cae40b71554d8 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 19:01:02.982378 22212 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4985530a07eb4edeaa3cae40b71554d8 [term 2 LEADER]: Becoming Leader. State: Replica: 4985530a07eb4edeaa3cae40b71554d8, State: Running, Role: LEADER
I20260811 19:01:02.983126 22212 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4985530a07eb4edeaa3cae40b71554d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4985530a07eb4edeaa3cae40b71554d8" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 37101 } }
I20260811 19:01:02.983619 22201 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4985530a07eb4edeaa3cae40b71554d8 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:01:02.987084 22215 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4985530a07eb4edeaa3cae40b71554d8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4985530a07eb4edeaa3cae40b71554d8. Latest consensus state: current_term: 2 leader_uuid: "4985530a07eb4edeaa3cae40b71554d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4985530a07eb4edeaa3cae40b71554d8" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 37101 } } }
I20260811 19:01:02.987835 22215 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4985530a07eb4edeaa3cae40b71554d8 [sys.catalog]: This master's current role is: LEADER
I20260811 19:01:02.992798 22214 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4985530a07eb4edeaa3cae40b71554d8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "4985530a07eb4edeaa3cae40b71554d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4985530a07eb4edeaa3cae40b71554d8" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 37101 } } }
I20260811 19:01:02.993333 22214 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4985530a07eb4edeaa3cae40b71554d8 [sys.catalog]: This master's current role is: LEADER
I20260811 19:01:02.997641 22223 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:01:03.013744 22223 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:01:03.016032 22223 catalog_manager.cc:1259] Loaded cluster ID: 3cfa2eb09efe418f8e8c0c1db4d80d41
I20260811 19:01:03.016186 22223 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:01:03.022188 22223 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260811 19:01:03.044179 22223 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260811 19:01:03.059816 22223 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:01:03.062513 22223 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 4985530a07eb4edeaa3cae40b71554d8: Loaded TSK: 0
I20260811 19:01:03.064447 22223 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 19:01:03.396162 22206 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:01:03.396780 22206 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:01:03.397008 22206 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:01:03.408162 22206 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:01:03.408313 22206 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:01:03.408493 22206 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.129
I20260811 19:01:03.423354 22206 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:41831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.129.keytab
--principal=kudu/127.29.112.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.129:41521
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.112.129
--webserver_port=37943
--tserver_master_addrs=127.29.112.190:37101
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.129
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:01:03.424853 22206 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:01:03.426290 22206 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:01:03.444792 22254 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:01:03.444265 22256 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:01:03.445235 22260 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:01:03.446442 22206 server_base.cc:1034] running on GCE node
I20260811 19:01:03.448079 22206 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:01:03.450374 22206 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:01:03.451825 22206 hybrid_clock.cc:648] HybridClock initialized: now 1786474863451689 us; error 80 us; skew 500 ppm
I20260811 19:01:03.452426 22206 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:01:03 dist-test-slave-h6n0 krb5kdc[21166](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474863, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:01:03.458500 22206 init.cc:377] Logged in from keytab as kudu/127.29.112.129@KRBTEST.COM (short username kudu)
I20260811 19:01:03.461081 22206 webserver.cc:460] Webserver started at http://127.29.112.129:37943/ using document root <none> and password file <none>
I20260811 19:01:03.462109 22206 fs_manager.cc:359] Metadata directory not provided
I20260811 19:01:03.462306 22206 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:01:03.471921 22206 fs_manager.cc:709] Time spent opening directory manager: real 0.007s user 0.006s sys 0.000s
I20260811 19:01:03.477640 22273 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:01:03.479964 22206 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260811 19:01:03.480202 22206 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-0/wal
uuid: "c5540dcd5d1846028c6ee159697fc086"
format_stamp: "Formatted at 2026-08-11 19:00:59 on dist-test-slave-h6n0"
server_key: "e368153fff9f1dc9903d82a309663ea9"
server_key_iv: "0ff1c8300acc5725394f2b0c52a33c6c"
server_key_version: "encryptionkey@0"
I20260811 19:01:03.481724 22206 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:01:03.506669 22206 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:01:03.513509 22206 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:01:03.513828 22206 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:01:03.516490 22206 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:01:03.520550 22206 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:01:03.520761 22206 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:01:03.520928 22206 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:01:03.521036 22206 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:01:03.578195 22206 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.129:41521
I20260811 19:01:03.578303 22457 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.129:41521 every 8 connection(s)
I20260811 19:01:03.580099 22206 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-0/data/info.pb
I20260811 19:01:03.586683 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 22206
I20260811 19:01:03.587777 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.130:46063
--local_ip_for_outbound_sockets=127.29.112.130
--tserver_master_addrs=127.29.112.190:37101
--webserver_port=40137
--webserver_interface=127.29.112.130
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.130.keytab
--principal=kudu/127.29.112.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:41831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 19:01:03 dist-test-slave-h6n0 krb5kdc[21166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474863, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.129@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 19:01:03.615973 22461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:03.584786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.129:42933 (local address 127.29.112.190:37101)
0811 19:01:03.585424 (+ 638us) server_negotiation.cc:206] Beginning negotiation
0811 19:01:03.585439 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 19:01:03.586475 (+ 1036us) server_negotiation.cc:407] Connection header received
0811 19:01:03.589241 (+ 2766us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:01:03.589278 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:01:03.589652 (+ 374us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:01:03.590038 (+ 386us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:01:03.591711 (+ 1673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:03.593443 (+ 1732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:03.595965 (+ 2522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:03.597570 (+ 1605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:03.597858 (+ 288us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:03.603154 (+ 5296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:01:03.603168 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:01:03.603195 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:01:03.603270 (+ 75us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:01:03.607855 (+ 4585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:01:03.609080 (+ 1225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:01:03.609093 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:01:03.609103 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:01:03.609345 (+ 242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:01:03.610926 (+ 1581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:01:03.610935 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:01:03.610941 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:01:03.611751 (+ 810us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:01:03.611993 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0811 19:01:03.613909 (+ 1916us) server_negotiation.cc:299] Negotiation successful
0811 19:01:03.614412 (+ 503us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":428,"thread_start_us":295,"threads_started":1}
I20260811 19:01:03.616138 22464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:03.585381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37101 (local address 127.29.112.129:42933)
0811 19:01:03.586251 (+ 870us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:03.586304 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 19:01:03.588598 (+ 2294us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:01:03.590188 (+ 1590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:01:03.590199 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:01:03.590921 (+ 722us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:01:03.591483 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:01:03.591501 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:01:03.593642 (+ 2141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:01:03.593650 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:01:03.595678 (+ 2028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:01:03.595697 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:01:03.597907 (+ 2210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:01:03.597917 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:01:03.598260 (+ 343us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:03.599195 (+ 935us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:01:03.599229 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:01:03.602781 (+ 3552us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:01:03.608133 (+ 5352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:01:03.608143 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:01:03.608157 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:01:03.608890 (+ 733us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:01:03.609621 (+ 731us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:01:03.609628 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:01:03.609632 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:01:03.610698 (+ 1066us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:01:03.612441 (+ 1743us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:01:03.612454 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:01:03.612994 (+ 540us) client_negotiation.cc:769] Sending connection context
0811 19:01:03.613565 (+ 571us) client_negotiation.cc:240] Negotiation successful
0811 19:01:03.614140 (+ 575us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":598,"thread_start_us":346,"threads_started":1}
I20260811 19:01:03.622574 22458 heartbeater.cc:347] Connected to a master server at 127.29.112.190:37101
I20260811 19:01:03.623417 22458 heartbeater.cc:464] Registering TS with master...
I20260811 19:01:03.624595 22458 heartbeater.cc:511] Master 127.29.112.190:37101 requested a full tablet report, sending...
I20260811 19:01:03.628924 22141 ts_manager.cc:194] Registered new tserver with Master: c5540dcd5d1846028c6ee159697fc086 (127.29.112.129:41521)
I20260811 19:01:03.632562 22141 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.129@KRBTEST.COM'} at 127.29.112.129:42933
W20260811 19:01:04.023693 22467 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:01:04.024080 22467 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:01:04.024263 22467 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:01:04.031896 22467 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:01:04.032074 22467 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:01:04.032258 22467 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.130
I20260811 19:01:04.046041 22467 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:41831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.130.keytab
--principal=kudu/127.29.112.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.130:46063
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.112.130
--webserver_port=40137
--tserver_master_addrs=127.29.112.190:37101
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.130
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:01:04.047640 22467 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:01:04.049085 22467 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:01:04.065234 22473 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:01:04.065315 22472 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:01:04.065268 22475 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:01:04.067011 22467 server_base.cc:1034] running on GCE node
I20260811 19:01:04.067852 22467 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:01:04.069078 22467 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:01:04.070335 22467 hybrid_clock.cc:648] HybridClock initialized: now 1786474864070290 us; error 43 us; skew 500 ppm
I20260811 19:01:04.070767 22467 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:01:04 dist-test-slave-h6n0 krb5kdc[21166](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474864, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:01:04.075536 22467 init.cc:377] Logged in from keytab as kudu/127.29.112.130@KRBTEST.COM (short username kudu)
I20260811 19:01:04.078051 22467 webserver.cc:460] Webserver started at http://127.29.112.130:40137/ using document root <none> and password file <none>
I20260811 19:01:04.079005 22467 fs_manager.cc:359] Metadata directory not provided
I20260811 19:01:04.079208 22467 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:01:04.087059 22467 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.001s sys 0.006s
I20260811 19:01:04.091578 22483 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:01:04.094354 22467 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.004s sys 0.000s
I20260811 19:01:04.094632 22467 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-1/wal
uuid: "ed49ffb15359470f84f5fae8705c97e4"
format_stamp: "Formatted at 2026-08-11 19:01:00 on dist-test-slave-h6n0"
server_key: "084c999a79f65c0a1b1a0cdf6a124b49"
server_key_iv: "7c1812b0d2ac818c81e65cb084c32743"
server_key_version: "encryptionkey@0"
I20260811 19:01:04.095762 22467 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:01:04.127955 22467 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:01:04.138193 22467 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:01:04.138607 22467 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:01:04.140607 22467 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:01:04.144289 22467 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:01:04.144536 22467 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:01:04.144701 22467 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:01:04.144798 22467 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:01:04.196903 22467 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.130:46063
I20260811 19:01:04.197062 22640 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.130:46063 every 8 connection(s)
I20260811 19:01:04.199666 22467 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-1/data/info.pb
I20260811 19:01:04.204972 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 22467
I20260811 19:01:04.206408 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.131:44959
--local_ip_for_outbound_sockets=127.29.112.131
--tserver_master_addrs=127.29.112.190:37101
--webserver_port=38097
--webserver_interface=127.29.112.131
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.131.keytab
--principal=kudu/127.29.112.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:41831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 19:01:04 dist-test-slave-h6n0 krb5kdc[21166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474864, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.130@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 19:01:04.231843 22643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:04.204076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.130:33055 (local address 127.29.112.190:37101)
0811 19:01:04.204775 (+ 699us) server_negotiation.cc:206] Beginning negotiation
0811 19:01:04.204781 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:01:04.206286 (+ 1505us) server_negotiation.cc:407] Connection header received
0811 19:01:04.209133 (+ 2847us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:01:04.209141 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:01:04.209267 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:01:04.209688 (+ 421us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:01:04.211988 (+ 2300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:04.212819 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:04.214670 (+ 1851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:04.215141 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:04.215303 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:04.224549 (+ 9246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:01:04.224556 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:01:04.224560 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:01:04.224608 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:01:04.227927 (+ 3319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:01:04.228972 (+ 1045us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:01:04.228977 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:01:04.228979 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:01:04.229127 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:01:04.229689 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:01:04.229694 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:01:04.229696 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:01:04.230028 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:01:04.230208 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0811 19:01:04.231128 (+ 920us) server_negotiation.cc:299] Negotiation successful
0811 19:01:04.231411 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":565,"thread_start_us":406,"threads_started":1}
I20260811 19:01:04.233040 22646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:04.204876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37101 (local address 127.29.112.130:33055)
0811 19:01:04.206031 (+ 1155us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:04.206101 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0811 19:01:04.208690 (+ 2589us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:01:04.210222 (+ 1532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:01:04.210233 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:01:04.211015 (+ 782us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:01:04.211710 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:01:04.211730 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:01:04.213033 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:01:04.213038 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:01:04.214451 (+ 1413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:01:04.214465 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:01:04.215451 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:01:04.215461 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:01:04.215718 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:04.216635 (+ 917us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:01:04.216668 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:01:04.223710 (+ 7042us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:01:04.228171 (+ 4461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:01:04.228185 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:01:04.228209 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:01:04.228789 (+ 580us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:01:04.229259 (+ 470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:01:04.229265 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:01:04.229269 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:01:04.229536 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:01:04.230179 (+ 643us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:01:04.230188 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:01:04.230703 (+ 515us) client_negotiation.cc:769] Sending connection context
0811 19:01:04.231022 (+ 319us) client_negotiation.cc:240] Negotiation successful
0811 19:01:04.231693 (+ 671us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":798,"thread_start_us":577,"threads_started":1}
I20260811 19:01:04.235889 22641 heartbeater.cc:347] Connected to a master server at 127.29.112.190:37101
I20260811 19:01:04.236482 22641 heartbeater.cc:464] Registering TS with master...
I20260811 19:01:04.237386 22641 heartbeater.cc:511] Master 127.29.112.190:37101 requested a full tablet report, sending...
I20260811 19:01:04.239682 22141 ts_manager.cc:194] Registered new tserver with Master: ed49ffb15359470f84f5fae8705c97e4 (127.29.112.130:46063)
I20260811 19:01:04.240708 22141 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.130@KRBTEST.COM'} at 127.29.112.130:33055
I20260811 19:01:04.637647 22458 heartbeater.cc:503] Master 127.29.112.190:37101 was elected leader, sending a full tablet report...
W20260811 19:01:04.688894 22647 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:01:04.689285 22647 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:01:04.689498 22647 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:01:04.698552 22647 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:01:04.698737 22647 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:01:04.698928 22647 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.131
I20260811 19:01:04.710274 22647 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:41831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.131.keytab
--principal=kudu/127.29.112.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.131:44959
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.112.131
--webserver_port=38097
--tserver_master_addrs=127.29.112.190:37101
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.131
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:01:04.711855 22647 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:01:04.713342 22647 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:01:04.736445 22659 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:01:04.736811 22656 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:01:04.736860 22657 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:01:04.738127 22647 server_base.cc:1034] running on GCE node
I20260811 19:01:04.738912 22647 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:01:04.740016 22647 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:01:04.741400 22647 hybrid_clock.cc:648] HybridClock initialized: now 1786474864741334 us; error 42 us; skew 500 ppm
I20260811 19:01:04.741789 22647 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:01:04 dist-test-slave-h6n0 krb5kdc[21166](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474864, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:01:04.745501 22647 init.cc:377] Logged in from keytab as kudu/127.29.112.131@KRBTEST.COM (short username kudu)
I20260811 19:01:04.747373 22647 webserver.cc:460] Webserver started at http://127.29.112.131:38097/ using document root <none> and password file <none>
I20260811 19:01:04.748150 22647 fs_manager.cc:359] Metadata directory not provided
I20260811 19:01:04.748324 22647 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:01:04.755628 22647 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 19:01:04.759500 22669 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:01:04.761449 22647 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 19:01:04.761787 22647 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-2/wal
uuid: "8b70c8cd4d2e4edea782997b0b2663aa"
format_stamp: "Formatted at 2026-08-11 19:01:00 on dist-test-slave-h6n0"
server_key: "17151f16d18aa4af7c494b1266623e68"
server_key_iv: "e7d77d4a59c2b5dcd2cca775b8306f50"
server_key_version: "encryptionkey@0"
I20260811 19:01:04.763123 22647 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:01:04.780990 22647 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:01:04.787580 22647 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:01:04.787899 22647 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:01:04.789116 22647 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:01:04.790665 22647 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:01:04.790796 22647 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:01:04.790930 22647 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:01:04.791033 22647 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:01:04.840878 22647 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.131:44959
I20260811 19:01:04.841144 22842 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.131:44959 every 8 connection(s)
I20260811 19:01:04.842615 22647 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-2/data/info.pb
I20260811 19:01:04.844523 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 22647
Aug 11 19:01:04 dist-test-slave-h6n0 krb5kdc[21166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474864, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.131@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 19:01:04.867321 22848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:04.846348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38352 (local address 127.29.112.190:37101)
0811 19:01:04.847173 (+ 825us) server_negotiation.cc:206] Beginning negotiation
0811 19:01:04.847285 (+ 112us) server_negotiation.cc:399] Waiting for connection header
0811 19:01:04.848870 (+ 1585us) server_negotiation.cc:407] Connection header received
0811 19:01:04.849150 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:01:04.849157 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:01:04.849290 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:01:04.849466 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:01:04.851844 (+ 2378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:04.853469 (+ 1625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:04.855325 (+ 1856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:04.856058 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:04.856234 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:04.860603 (+ 4369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:01:04.860609 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:01:04.860612 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:01:04.860681 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:01:04.864178 (+ 3497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:01:04.864994 (+ 816us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:01:04.864998 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:01:04.865001 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:01:04.865138 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:01:04.865575 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:01:04.865578 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:01:04.865580 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:01:04.865896 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:01:04.866260 (+ 364us) server_negotiation.cc:1035] Waiting for connection context
0811 19:01:04.866639 (+ 379us) server_negotiation.cc:299] Negotiation successful
0811 19:01:04.866877 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":599,"spinlock_wait_cycles":7808,"thread_start_us":394,"threads_started":1}
I20260811 19:01:04.873667 22850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:04.847406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.131:36723 (local address 127.29.112.190:37101)
0811 19:01:04.848188 (+ 782us) server_negotiation.cc:206] Beginning negotiation
0811 19:01:04.848198 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 19:01:04.849828 (+ 1630us) server_negotiation.cc:407] Connection header received
0811 19:01:04.853737 (+ 3909us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:01:04.853743 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:01:04.853881 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:01:04.854020 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:01:04.856443 (+ 2423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:04.858036 (+ 1593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:04.859968 (+ 1932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:04.860462 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:04.860601 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:04.865807 (+ 5206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:01:04.865813 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:01:04.865817 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:01:04.865872 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:01:04.869797 (+ 3925us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:01:04.870952 (+ 1155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:01:04.870956 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:01:04.870959 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:01:04.871103 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:01:04.871605 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:01:04.871610 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:01:04.871612 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:01:04.872061 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:01:04.872236 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0811 19:01:04.872983 (+ 747us) server_negotiation.cc:299] Negotiation successful
0811 19:01:04.873250 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":593,"spinlock_wait_cycles":1280,"thread_start_us":375,"threads_started":1}
I20260811 19:01:04.874941 22851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:04.848390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37101 (local address 127.29.112.131:36723)
0811 19:01:04.849622 (+ 1232us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:04.849712 (+ 90us) client_negotiation.cc:174] Beginning negotiation
0811 19:01:04.853302 (+ 3590us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:01:04.854365 (+ 1063us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:01:04.854376 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:01:04.855097 (+ 721us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:01:04.856025 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:01:04.856054 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:01:04.858341 (+ 2287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:01:04.858346 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:01:04.859776 (+ 1430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:01:04.859795 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:01:04.860721 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:01:04.860731 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:01:04.861011 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:04.862255 (+ 1244us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:01:04.862306 (+ 51us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:01:04.865380 (+ 3074us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:01:04.870038 (+ 4658us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:01:04.870049 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:01:04.870066 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:01:04.870794 (+ 728us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:01:04.871227 (+ 433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:01:04.871232 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:01:04.871236 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:01:04.871451 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:01:04.872210 (+ 759us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:01:04.872218 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:01:04.872609 (+ 391us) client_negotiation.cc:769] Sending connection context
0811 19:01:04.872886 (+ 277us) client_negotiation.cc:240] Negotiation successful
0811 19:01:04.873479 (+ 593us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":786,"thread_start_us":473,"threads_started":1}
I20260811 19:01:04.876654 22843 heartbeater.cc:347] Connected to a master server at 127.29.112.190:37101
I20260811 19:01:04.877239 22843 heartbeater.cc:464] Registering TS with master...
I20260811 19:01:04.878154 22843 heartbeater.cc:511] Master 127.29.112.190:37101 requested a full tablet report, sending...
I20260811 19:01:04.879844 22141 ts_manager.cc:194] Registered new tserver with Master: 8b70c8cd4d2e4edea782997b0b2663aa (127.29.112.131:44959)
I20260811 19:01:04.880708 22141 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.131@KRBTEST.COM'} at 127.29.112.131:36723
I20260811 19:01:04.886680 30146 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 19:01:04.901024 22850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:04.888726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38362 (local address 127.29.112.190:37101)
0811 19:01:04.888944 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0811 19:01:04.888949 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:01:04.889602 (+ 653us) server_negotiation.cc:407] Connection header received
0811 19:01:04.889804 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:01:04.889810 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:01:04.890016 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:01:04.890168 (+ 152us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:01:04.891872 (+ 1704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:04.893502 (+ 1630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:04.897415 (+ 3913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:04.898469 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:04.898686 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:04.899357 (+ 671us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:01:04.900151 (+ 794us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:01:04.900304 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 19:01:04.900610 (+ 306us) server_negotiation.cc:299] Negotiation successful
0811 19:01:04.900706 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260811 19:01:04.905449 22141 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:38362:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 19:01:04.917927 22141 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 19:01:04.951020 22866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:04.940081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40122 (local address 127.29.112.129:41521)
0811 19:01:04.940764 (+ 683us) server_negotiation.cc:206] Beginning negotiation
0811 19:01:04.940776 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 19:01:04.941308 (+ 532us) server_negotiation.cc:407] Connection header received
0811 19:01:04.941912 (+ 604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:01:04.941922 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:01:04.942204 (+ 282us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:01:04.942394 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:01:04.943652 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:04.944882 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:04.948588 (+ 3706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:04.949740 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:04.949995 (+ 255us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:04.950185 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0811 19:01:04.950474 (+ 289us) server_negotiation.cc:299] Negotiation successful
0811 19:01:04.950663 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":493,"thread_start_us":311,"threads_started":1}
I20260811 19:01:04.951086 22863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:04.939740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:41521 (local address 127.0.0.1:40122)
0811 19:01:04.941144 (+ 1404us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:04.941195 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 19:01:04.941629 (+ 434us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:01:04.942776 (+ 1147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:01:04.942784 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:01:04.942860 (+ 76us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:01:04.943425 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:01:04.943436 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:01:04.945046 (+ 1610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:01:04.945050 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:01:04.948123 (+ 3073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:01:04.948147 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:01:04.949995 (+ 1848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:01:04.950003 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:01:04.950194 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:04.950257 (+ 63us) client_negotiation.cc:769] Sending connection context
0811 19:01:04.950419 (+ 162us) client_negotiation.cc:240] Negotiation successful
0811 19:01:04.950653 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1147,"mutex_wait_us":125,"spinlock_wait_cycles":896,"thread_start_us":898,"threads_started":1}
I20260811 19:01:04.952847 22867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:04.940306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.131:44959 (local address 127.0.0.1:38906)
0811 19:01:04.941706 (+ 1400us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:04.941726 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 19:01:04.941860 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:01:04.942562 (+ 702us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:01:04.942570 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:01:04.942599 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:01:04.943212 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:01:04.943231 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:01:04.945606 (+ 2375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:01:04.945617 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:01:04.948917 (+ 3300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:01:04.948939 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:01:04.951964 (+ 3025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:01:04.951971 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:01:04.952136 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:04.952168 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 19:01:04.952271 (+ 103us) client_negotiation.cc:240] Negotiation successful
0811 19:01:04.952404 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1277,"mutex_wait_us":71,"spinlock_wait_cycles":2304,"thread_start_us":2347,"threads_started":1}
I20260811 19:01:04.953428 22868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:04.940587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38906 (local address 127.29.112.131:44959)
0811 19:01:04.941321 (+ 734us) server_negotiation.cc:206] Beginning negotiation
0811 19:01:04.941331 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 19:01:04.941903 (+ 572us) server_negotiation.cc:407] Connection header received
0811 19:01:04.942028 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:01:04.942036 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:01:04.942293 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:01:04.943428 (+ 1135us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:01:04.943699 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:04.945447 (+ 1748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:04.949337 (+ 3890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:04.951613 (+ 2276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:04.951939 (+ 326us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:04.952136 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0811 19:01:04.952686 (+ 550us) server_negotiation.cc:299] Negotiation successful
0811 19:01:04.952886 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":557,"thread_start_us":397,"threads_started":1}
I20260811 19:01:04.953883 22869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:04.941817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.130:46063 (local address 127.0.0.1:56190)
0811 19:01:04.942395 (+ 578us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:04.942420 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 19:01:04.942562 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:01:04.944054 (+ 1492us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:01:04.944059 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:01:04.944084 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:01:04.944475 (+ 391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:01:04.944498 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:01:04.946305 (+ 1807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:01:04.946311 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:01:04.948879 (+ 2568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:01:04.948901 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:01:04.953142 (+ 4241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:01:04.953151 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:01:04.953336 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:04.953383 (+ 47us) client_negotiation.cc:769] Sending connection context
0811 19:01:04.953468 (+ 85us) client_negotiation.cc:240] Negotiation successful
0811 19:01:04.953606 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":461,"mutex_wait_us":35,"thread_start_us":782,"threads_started":1}
I20260811 19:01:04.954880 22871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:04.942430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56190 (local address 127.29.112.130:46063)
0811 19:01:04.943444 (+ 1014us) server_negotiation.cc:206] Beginning negotiation
0811 19:01:04.943453 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 19:01:04.943479 (+ 26us) server_negotiation.cc:407] Connection header received
0811 19:01:04.943588 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:01:04.943596 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:01:04.943827 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:01:04.944683 (+ 856us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:01:04.944898 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:04.946165 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:04.949128 (+ 2963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:04.952867 (+ 3739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:04.954160 (+ 1293us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:04.954276 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 19:01:04.954333 (+ 57us) server_negotiation.cc:299] Negotiation successful
0811 19:01:04.954569 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":854,"thread_start_us":1466,"threads_started":1}
I20260811 19:01:04.956303 22355 tablet_service.cc:1467] Processing CreateTablet for tablet 8797d08a7ebb48e6b93ea015e240b5bf (DEFAULT_TABLE table=test-table [id=d5bafed92c414807b1d543932a1e0851]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:01:04.956614 22743 tablet_service.cc:1467] Processing CreateTablet for tablet 8797d08a7ebb48e6b93ea015e240b5bf (DEFAULT_TABLE table=test-table [id=d5bafed92c414807b1d543932a1e0851]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:01:04.957755 22355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8797d08a7ebb48e6b93ea015e240b5bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:01:04.957973 22743 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8797d08a7ebb48e6b93ea015e240b5bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:01:04.958637 22549 tablet_service.cc:1467] Processing CreateTablet for tablet 8797d08a7ebb48e6b93ea015e240b5bf (DEFAULT_TABLE table=test-table [id=d5bafed92c414807b1d543932a1e0851]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:01:04.959956 22549 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8797d08a7ebb48e6b93ea015e240b5bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:01:04.971033 22872 tablet_bootstrap.cc:492] T 8797d08a7ebb48e6b93ea015e240b5bf P c5540dcd5d1846028c6ee159697fc086: Bootstrap starting.
I20260811 19:01:04.974998 22876 tablet_bootstrap.cc:492] T 8797d08a7ebb48e6b93ea015e240b5bf P 8b70c8cd4d2e4edea782997b0b2663aa: Bootstrap starting.
I20260811 19:01:04.975768 22872 tablet_bootstrap.cc:654] T 8797d08a7ebb48e6b93ea015e240b5bf P c5540dcd5d1846028c6ee159697fc086: Neither blocks nor log segments found. Creating new log.
I20260811 19:01:04.977151 22877 tablet_bootstrap.cc:492] T 8797d08a7ebb48e6b93ea015e240b5bf P ed49ffb15359470f84f5fae8705c97e4: Bootstrap starting.
I20260811 19:01:04.977152 22872 log.cc:824] T 8797d08a7ebb48e6b93ea015e240b5bf P c5540dcd5d1846028c6ee159697fc086: Log is configured to *not* fsync() on all Append() calls
I20260811 19:01:04.979105 22876 tablet_bootstrap.cc:654] T 8797d08a7ebb48e6b93ea015e240b5bf P 8b70c8cd4d2e4edea782997b0b2663aa: Neither blocks nor log segments found. Creating new log.
I20260811 19:01:04.980540 22876 log.cc:824] T 8797d08a7ebb48e6b93ea015e240b5bf P 8b70c8cd4d2e4edea782997b0b2663aa: Log is configured to *not* fsync() on all Append() calls
I20260811 19:01:04.980844 22877 tablet_bootstrap.cc:654] T 8797d08a7ebb48e6b93ea015e240b5bf P ed49ffb15359470f84f5fae8705c97e4: Neither blocks nor log segments found. Creating new log.
I20260811 19:01:04.981415 22872 tablet_bootstrap.cc:492] T 8797d08a7ebb48e6b93ea015e240b5bf P c5540dcd5d1846028c6ee159697fc086: No bootstrap required, opened a new log
I20260811 19:01:04.981803 22872 ts_tablet_manager.cc:1397] T 8797d08a7ebb48e6b93ea015e240b5bf P c5540dcd5d1846028c6ee159697fc086: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260811 19:01:04.982935 22877 log.cc:824] T 8797d08a7ebb48e6b93ea015e240b5bf P ed49ffb15359470f84f5fae8705c97e4: Log is configured to *not* fsync() on all Append() calls
I20260811 19:01:04.985165 22876 tablet_bootstrap.cc:492] T 8797d08a7ebb48e6b93ea015e240b5bf P 8b70c8cd4d2e4edea782997b0b2663aa: No bootstrap required, opened a new log
I20260811 19:01:04.985510 22876 ts_tablet_manager.cc:1397] T 8797d08a7ebb48e6b93ea015e240b5bf P 8b70c8cd4d2e4edea782997b0b2663aa: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.003s
I20260811 19:01:04.986354 22877 tablet_bootstrap.cc:492] T 8797d08a7ebb48e6b93ea015e240b5bf P ed49ffb15359470f84f5fae8705c97e4: No bootstrap required, opened a new log
I20260811 19:01:04.986663 22877 ts_tablet_manager.cc:1397] T 8797d08a7ebb48e6b93ea015e240b5bf P ed49ffb15359470f84f5fae8705c97e4: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260811 19:01:04.991030 22872 raft_consensus.cc:348] T 8797d08a7ebb48e6b93ea015e240b5bf P c5540dcd5d1846028c6ee159697fc086 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5540dcd5d1846028c6ee159697fc086" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 41521 } } peers { permanent_uuid: "8b70c8cd4d2e4edea782997b0b2663aa" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 44959 } } peers { permanent_uuid: "ed49ffb15359470f84f5fae8705c97e4" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 46063 } }
I20260811 19:01:04.991443 22872 raft_consensus.cc:374] T 8797d08a7ebb48e6b93ea015e240b5bf P c5540dcd5d1846028c6ee159697fc086 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:01:04.991587 22872 raft_consensus.cc:729] T 8797d08a7ebb48e6b93ea015e240b5bf P c5540dcd5d1846028c6ee159697fc086 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5540dcd5d1846028c6ee159697fc086, State: Initialized, Role: FOLLOWER
I20260811 19:01:04.992309 22872 consensus_queue.cc:260] T 8797d08a7ebb48e6b93ea015e240b5bf P c5540dcd5d1846028c6ee159697fc086 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5540dcd5d1846028c6ee159697fc086" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 41521 } } peers { permanent_uuid: "8b70c8cd4d2e4edea782997b0b2663aa" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 44959 } } peers { permanent_uuid: "ed49ffb15359470f84f5fae8705c97e4" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 46063 } }
I20260811 19:01:04.996389 22872 ts_tablet_manager.cc:1428] T 8797d08a7ebb48e6b93ea015e240b5bf P c5540dcd5d1846028c6ee159697fc086: Time spent starting tablet: real 0.014s user 0.012s sys 0.004s
I20260811 19:01:04.998436 22876 raft_consensus.cc:348] T 8797d08a7ebb48e6b93ea015e240b5bf P 8b70c8cd4d2e4edea782997b0b2663aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5540dcd5d1846028c6ee159697fc086" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 41521 } } peers { permanent_uuid: "8b70c8cd4d2e4edea782997b0b2663aa" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 44959 } } peers { permanent_uuid: "ed49ffb15359470f84f5fae8705c97e4" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 46063 } }
I20260811 19:01:04.998698 22877 raft_consensus.cc:348] T 8797d08a7ebb48e6b93ea015e240b5bf P ed49ffb15359470f84f5fae8705c97e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5540dcd5d1846028c6ee159697fc086" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 41521 } } peers { permanent_uuid: "8b70c8cd4d2e4edea782997b0b2663aa" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 44959 } } peers { permanent_uuid: "ed49ffb15359470f84f5fae8705c97e4" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 46063 } }
I20260811 19:01:04.999058 22876 raft_consensus.cc:374] T 8797d08a7ebb48e6b93ea015e240b5bf P 8b70c8cd4d2e4edea782997b0b2663aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:01:04.999202 22877 raft_consensus.cc:374] T 8797d08a7ebb48e6b93ea015e240b5bf P ed49ffb15359470f84f5fae8705c97e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:01:04.999229 22876 raft_consensus.cc:729] T 8797d08a7ebb48e6b93ea015e240b5bf P 8b70c8cd4d2e4edea782997b0b2663aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b70c8cd4d2e4edea782997b0b2663aa, State: Initialized, Role: FOLLOWER
I20260811 19:01:04.999312 22877 raft_consensus.cc:729] T 8797d08a7ebb48e6b93ea015e240b5bf P ed49ffb15359470f84f5fae8705c97e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed49ffb15359470f84f5fae8705c97e4, State: Initialized, Role: FOLLOWER
I20260811 19:01:04.999977 22877 consensus_queue.cc:260] T 8797d08a7ebb48e6b93ea015e240b5bf P ed49ffb15359470f84f5fae8705c97e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5540dcd5d1846028c6ee159697fc086" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 41521 } } peers { permanent_uuid: "8b70c8cd4d2e4edea782997b0b2663aa" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 44959 } } peers { permanent_uuid: "ed49ffb15359470f84f5fae8705c97e4" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 46063 } }
I20260811 19:01:04.999989 22876 consensus_queue.cc:260] T 8797d08a7ebb48e6b93ea015e240b5bf P 8b70c8cd4d2e4edea782997b0b2663aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5540dcd5d1846028c6ee159697fc086" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 41521 } } peers { permanent_uuid: "8b70c8cd4d2e4edea782997b0b2663aa" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 44959 } } peers { permanent_uuid: "ed49ffb15359470f84f5fae8705c97e4" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 46063 } }
I20260811 19:01:05.002127 22641 heartbeater.cc:503] Master 127.29.112.190:37101 was elected leader, sending a full tablet report...
I20260811 19:01:05.002128 22843 heartbeater.cc:503] Master 127.29.112.190:37101 was elected leader, sending a full tablet report...
I20260811 19:01:05.002604 22877 ts_tablet_manager.cc:1428] T 8797d08a7ebb48e6b93ea015e240b5bf P ed49ffb15359470f84f5fae8705c97e4: Time spent starting tablet: real 0.016s user 0.014s sys 0.000s
I20260811 19:01:05.002702 22876 ts_tablet_manager.cc:1428] T 8797d08a7ebb48e6b93ea015e240b5bf P 8b70c8cd4d2e4edea782997b0b2663aa: Time spent starting tablet: real 0.017s user 0.017s sys 0.000s
W20260811 19:01:05.083587 22460 tablet.cc:2369] T 8797d08a7ebb48e6b93ea015e240b5bf P c5540dcd5d1846028c6ee159697fc086: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 19:01:05.094818 22844 tablet.cc:2369] T 8797d08a7ebb48e6b93ea015e240b5bf P 8b70c8cd4d2e4edea782997b0b2663aa: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 19:01:05.122997 22884 raft_consensus.cc:482] T 8797d08a7ebb48e6b93ea015e240b5bf P 8b70c8cd4d2e4edea782997b0b2663aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:01:05.123323 22884 raft_consensus.cc:504] T 8797d08a7ebb48e6b93ea015e240b5bf P 8b70c8cd4d2e4edea782997b0b2663aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5540dcd5d1846028c6ee159697fc086" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 41521 } } peers { permanent_uuid: "8b70c8cd4d2e4edea782997b0b2663aa" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 44959 } } peers { permanent_uuid: "ed49ffb15359470f84f5fae8705c97e4" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 46063 } }
I20260811 19:01:05.125499 22884 leader_election.cc:288] T 8797d08a7ebb48e6b93ea015e240b5bf P 8b70c8cd4d2e4edea782997b0b2663aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c5540dcd5d1846028c6ee159697fc086 (127.29.112.129:41521), ed49ffb15359470f84f5fae8705c97e4 (127.29.112.130:46063)
I20260811 19:01:05.131520 22866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:05.126055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.131:43839 (local address 127.29.112.129:41521)
0811 19:01:05.126282 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0811 19:01:05.126288 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:01:05.126310 (+ 22us) server_negotiation.cc:407] Connection header received
0811 19:01:05.126503 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:01:05.126512 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:01:05.126661 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:01:05.126793 (+ 132us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:01:05.127355 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:05.128088 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:05.129697 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:05.130466 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:05.130696 (+ 230us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:05.130771 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 19:01:05.131114 (+ 343us) server_negotiation.cc:299] Negotiation successful
0811 19:01:05.131239 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 19:01:05.131531 22851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:05.125769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:41521 (local address 127.29.112.131:43839)
0811 19:01:05.126089 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:05.126133 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 19:01:05.126287 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:01:05.127004 (+ 717us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:01:05.127008 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:01:05.127043 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:01:05.127203 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:01:05.127210 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:01:05.128230 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:01:05.128236 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:01:05.129515 (+ 1279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:01:05.129527 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:01:05.130688 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:01:05.130694 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:01:05.130936 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:05.130965 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 19:01:05.131076 (+ 111us) client_negotiation.cc:240] Negotiation successful
0811 19:01:05.131221 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"mutex_wait_us":37}
I20260811 19:01:05.133126 22387 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8797d08a7ebb48e6b93ea015e240b5bf" candidate_uuid: "8b70c8cd4d2e4edea782997b0b2663aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5540dcd5d1846028c6ee159697fc086" is_pre_election: true
I20260811 19:01:05.133783 22387 raft_consensus.cc:2454] T 8797d08a7ebb48e6b93ea015e240b5bf P c5540dcd5d1846028c6ee159697fc086 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b70c8cd4d2e4edea782997b0b2663aa in term 0.
I20260811 19:01:05.133966 22871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:05.126853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.131:50035 (local address 127.29.112.130:46063)
0811 19:01:05.127078 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 19:01:05.127085 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:01:05.127281 (+ 196us) server_negotiation.cc:407] Connection header received
0811 19:01:05.127474 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:01:05.127479 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:01:05.127545 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:01:05.127643 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:01:05.128168 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:05.128851 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:05.131621 (+ 2770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:05.132846 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:05.133076 (+ 230us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:05.133160 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 19:01:05.133461 (+ 301us) server_negotiation.cc:299] Negotiation successful
0811 19:01:05.133636 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260811 19:01:05.134120 22889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:05.126317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.130:46063 (local address 127.29.112.131:50035)
0811 19:01:05.127180 (+ 863us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:05.127194 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 19:01:05.127331 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:01:05.127661 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:01:05.127665 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:01:05.127687 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:01:05.127996 (+ 309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:01:05.128004 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:01:05.129006 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:01:05.129011 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:01:05.131292 (+ 2281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:01:05.131321 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:01:05.133089 (+ 1768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:01:05.133095 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:01:05.133281 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:05.133309 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 19:01:05.133429 (+ 120us) client_negotiation.cc:240] Negotiation successful
0811 19:01:05.133651 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":737,"mutex_wait_us":34,"thread_start_us":482,"threads_started":1}
I20260811 19:01:05.134876 22670 leader_election.cc:302] T 8797d08a7ebb48e6b93ea015e240b5bf P 8b70c8cd4d2e4edea782997b0b2663aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b70c8cd4d2e4edea782997b0b2663aa, c5540dcd5d1846028c6ee159697fc086; no voters:
I20260811 19:01:05.135465 22884 raft_consensus.cc:2781] T 8797d08a7ebb48e6b93ea015e240b5bf P 8b70c8cd4d2e4edea782997b0b2663aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 19:01:05.135686 22884 raft_consensus.cc:482] T 8797d08a7ebb48e6b93ea015e240b5bf P 8b70c8cd4d2e4edea782997b0b2663aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 19:01:05.135814 22884 raft_consensus.cc:3037] T 8797d08a7ebb48e6b93ea015e240b5bf P 8b70c8cd4d2e4edea782997b0b2663aa [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:01:05.135818 22575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8797d08a7ebb48e6b93ea015e240b5bf" candidate_uuid: "8b70c8cd4d2e4edea782997b0b2663aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed49ffb15359470f84f5fae8705c97e4" is_pre_election: true
I20260811 19:01:05.136407 22575 raft_consensus.cc:2454] T 8797d08a7ebb48e6b93ea015e240b5bf P ed49ffb15359470f84f5fae8705c97e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b70c8cd4d2e4edea782997b0b2663aa in term 0.
I20260811 19:01:05.138422 22884 raft_consensus.cc:504] T 8797d08a7ebb48e6b93ea015e240b5bf P 8b70c8cd4d2e4edea782997b0b2663aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5540dcd5d1846028c6ee159697fc086" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 41521 } } peers { permanent_uuid: "8b70c8cd4d2e4edea782997b0b2663aa" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 44959 } } peers { permanent_uuid: "ed49ffb15359470f84f5fae8705c97e4" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 46063 } }
I20260811 19:01:05.139187 22884 leader_election.cc:288] T 8797d08a7ebb48e6b93ea015e240b5bf P 8b70c8cd4d2e4edea782997b0b2663aa [CANDIDATE]: Term 1 election: Requested vote from peers c5540dcd5d1846028c6ee159697fc086 (127.29.112.129:41521), ed49ffb15359470f84f5fae8705c97e4 (127.29.112.130:46063)
I20260811 19:01:05.139958 22387 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8797d08a7ebb48e6b93ea015e240b5bf" candidate_uuid: "8b70c8cd4d2e4edea782997b0b2663aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5540dcd5d1846028c6ee159697fc086"
I20260811 19:01:05.139976 22575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8797d08a7ebb48e6b93ea015e240b5bf" candidate_uuid: "8b70c8cd4d2e4edea782997b0b2663aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed49ffb15359470f84f5fae8705c97e4"
I20260811 19:01:05.140185 22575 raft_consensus.cc:3037] T 8797d08a7ebb48e6b93ea015e240b5bf P ed49ffb15359470f84f5fae8705c97e4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:01:05.140199 22387 raft_consensus.cc:3037] T 8797d08a7ebb48e6b93ea015e240b5bf P c5540dcd5d1846028c6ee159697fc086 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:01:05.142625 22387 raft_consensus.cc:2454] T 8797d08a7ebb48e6b93ea015e240b5bf P c5540dcd5d1846028c6ee159697fc086 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b70c8cd4d2e4edea782997b0b2663aa in term 1.
I20260811 19:01:05.142624 22575 raft_consensus.cc:2454] T 8797d08a7ebb48e6b93ea015e240b5bf P ed49ffb15359470f84f5fae8705c97e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b70c8cd4d2e4edea782997b0b2663aa in term 1.
I20260811 19:01:05.143410 22670 leader_election.cc:302] T 8797d08a7ebb48e6b93ea015e240b5bf P 8b70c8cd4d2e4edea782997b0b2663aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b70c8cd4d2e4edea782997b0b2663aa, c5540dcd5d1846028c6ee159697fc086; no voters:
I20260811 19:01:05.143894 22884 raft_consensus.cc:2781] T 8797d08a7ebb48e6b93ea015e240b5bf P 8b70c8cd4d2e4edea782997b0b2663aa [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:01:05.144769 22884 raft_consensus.cc:686] T 8797d08a7ebb48e6b93ea015e240b5bf P 8b70c8cd4d2e4edea782997b0b2663aa [term 1 LEADER]: Becoming Leader. State: Replica: 8b70c8cd4d2e4edea782997b0b2663aa, State: Running, Role: LEADER
I20260811 19:01:05.145354 22884 consensus_queue.cc:237] T 8797d08a7ebb48e6b93ea015e240b5bf P 8b70c8cd4d2e4edea782997b0b2663aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5540dcd5d1846028c6ee159697fc086" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 41521 } } peers { permanent_uuid: "8b70c8cd4d2e4edea782997b0b2663aa" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 44959 } } peers { permanent_uuid: "ed49ffb15359470f84f5fae8705c97e4" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 46063 } }
I20260811 19:01:05.150822 22141 catalog_manager.cc:5495] T 8797d08a7ebb48e6b93ea015e240b5bf P 8b70c8cd4d2e4edea782997b0b2663aa reported cstate change: term changed from 0 to 1, leader changed from <none> to 8b70c8cd4d2e4edea782997b0b2663aa (127.29.112.131). New cstate: current_term: 1 leader_uuid: "8b70c8cd4d2e4edea782997b0b2663aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5540dcd5d1846028c6ee159697fc086" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 41521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b70c8cd4d2e4edea782997b0b2663aa" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 44959 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ed49ffb15359470f84f5fae8705c97e4" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 46063 } health_report { overall_health: UNKNOWN } } }
W20260811 19:01:05.202884 22642 tablet.cc:2369] T 8797d08a7ebb48e6b93ea015e240b5bf P ed49ffb15359470f84f5fae8705c97e4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 19:01:05.204339 22868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:05.196954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38922 (local address 127.29.112.131:44959)
0811 19:01:05.197194 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0811 19:01:05.197199 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:01:05.197218 (+ 19us) server_negotiation.cc:407] Connection header received
0811 19:01:05.197567 (+ 349us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:01:05.197572 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:01:05.197805 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:01:05.198001 (+ 196us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:01:05.198634 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:05.199669 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:05.201759 (+ 2090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:05.202205 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:05.202317 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:05.202939 (+ 622us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:01:05.203659 (+ 720us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:01:05.203796 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 19:01:05.203943 (+ 147us) server_negotiation.cc:299] Negotiation successful
0811 19:01:05.204043 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 19:01:05.216616 22387 raft_consensus.cc:1261] T 8797d08a7ebb48e6b93ea015e240b5bf P c5540dcd5d1846028c6ee159697fc086 [term 1 FOLLOWER]: Refusing update from remote peer 8b70c8cd4d2e4edea782997b0b2663aa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:01:05.216984 22575 raft_consensus.cc:1261] T 8797d08a7ebb48e6b93ea015e240b5bf P ed49ffb15359470f84f5fae8705c97e4 [term 1 FOLLOWER]: Refusing update from remote peer 8b70c8cd4d2e4edea782997b0b2663aa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:01:05.218185 22884 consensus_queue.cc:1035] T 8797d08a7ebb48e6b93ea015e240b5bf P 8b70c8cd4d2e4edea782997b0b2663aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5540dcd5d1846028c6ee159697fc086" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 41521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:01:05.218657 22890 consensus_queue.cc:1035] T 8797d08a7ebb48e6b93ea015e240b5bf P 8b70c8cd4d2e4edea782997b0b2663aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed49ffb15359470f84f5fae8705c97e4" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 46063 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:01:05.233011 22895 mvcc.cc:205] Tried to move back new op lower bound from 7317401047912058880 to 7317401047639556096. Current Snapshot: MvccSnapshot[applied={T|T < 7317401047912058880}]
I20260811 19:01:05.254407 22141 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:38362:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:01:05.254858 22141 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:38362:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:01:05.260605 22141 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4985530a07eb4edeaa3cae40b71554d8: Sending DeleteTablet for 3 replicas of tablet 8797d08a7ebb48e6b93ea015e240b5bf
I20260811 19:01:05.262313 22355 tablet_service.cc:1514] Processing DeleteTablet for tablet 8797d08a7ebb48e6b93ea015e240b5bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:01:05 UTC) from {username='kudu', principal='kudu/127.29.112.190@KRBTEST.COM'} at 127.0.0.1:40122
I20260811 19:01:05.262377 22743 tablet_service.cc:1514] Processing DeleteTablet for tablet 8797d08a7ebb48e6b93ea015e240b5bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:01:05 UTC) from {username='kudu', principal='kudu/127.29.112.190@KRBTEST.COM'} at 127.0.0.1:38906
I20260811 19:01:05.262377 22549 tablet_service.cc:1514] Processing DeleteTablet for tablet 8797d08a7ebb48e6b93ea015e240b5bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:01:05 UTC) from {username='kudu', principal='kudu/127.29.112.190@KRBTEST.COM'} at 127.0.0.1:56190
I20260811 19:01:05.264081 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 22206
I20260811 19:01:05.274811 22915 tablet_replica.cc:330] T 8797d08a7ebb48e6b93ea015e240b5bf P ed49ffb15359470f84f5fae8705c97e4: stopping tablet replica
I20260811 19:01:05.276213 22915 raft_consensus.cc:2229] T 8797d08a7ebb48e6b93ea015e240b5bf P ed49ffb15359470f84f5fae8705c97e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:01:05.277171 22915 raft_consensus.cc:2258] T 8797d08a7ebb48e6b93ea015e240b5bf P ed49ffb15359470f84f5fae8705c97e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:01:05.279719 22918 tablet_replica.cc:330] T 8797d08a7ebb48e6b93ea015e240b5bf P 8b70c8cd4d2e4edea782997b0b2663aa: stopping tablet replica
I20260811 19:01:05.280146 22916 tablet_replica.cc:330] T 8797d08a7ebb48e6b93ea015e240b5bf P c5540dcd5d1846028c6ee159697fc086: stopping tablet replica
I20260811 19:01:05.280540 22915 ts_tablet_manager.cc:1905] T 8797d08a7ebb48e6b93ea015e240b5bf P ed49ffb15359470f84f5fae8705c97e4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:01:05.281085 22918 raft_consensus.cc:2229] T 8797d08a7ebb48e6b93ea015e240b5bf P 8b70c8cd4d2e4edea782997b0b2663aa [term 1 LEADER]: Raft consensus shutting down.
I20260811 19:01:05.281508 22916 raft_consensus.cc:2229] T 8797d08a7ebb48e6b93ea015e240b5bf P c5540dcd5d1846028c6ee159697fc086 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:01:05.282306 22918 raft_consensus.cc:2258] T 8797d08a7ebb48e6b93ea015e240b5bf P 8b70c8cd4d2e4edea782997b0b2663aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:01:05.283011 22916 raft_consensus.cc:2258] T 8797d08a7ebb48e6b93ea015e240b5bf P c5540dcd5d1846028c6ee159697fc086 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:01:05.286636 22916 ts_tablet_manager.cc:1905] T 8797d08a7ebb48e6b93ea015e240b5bf P c5540dcd5d1846028c6ee159697fc086: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:01:05.293184 22918 ts_tablet_manager.cc:1905] T 8797d08a7ebb48e6b93ea015e240b5bf P 8b70c8cd4d2e4edea782997b0b2663aa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:01:05.301626 22916 ts_tablet_manager.cc:1918] T 8797d08a7ebb48e6b93ea015e240b5bf P c5540dcd5d1846028c6ee159697fc086: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:01:05.301668 22866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:05.268486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40130 (local address 127.29.112.129:41521)
0811 19:01:05.268729 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0811 19:01:05.268738 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 19:01:05.268764 (+ 26us) server_negotiation.cc:407] Connection header received
0811 19:01:05.268861 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:01:05.268870 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:01:05.268993 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:01:05.269334 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:01:05.274965 (+ 5631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:05.277051 (+ 2086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:05.279870 (+ 2819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:05.280600 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:05.280714 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:05.286288 (+ 5574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:01:05.286306 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:01:05.286325 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:01:05.286477 (+ 152us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:01:05.294501 (+ 8024us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:01:05.296479 (+ 1978us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:01:05.296495 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:01:05.296500 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:01:05.296915 (+ 415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:01:05.297690 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:01:05.297695 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:01:05.297697 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:01:05.298840 (+ 1143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:01:05.299318 (+ 478us) server_negotiation.cc:1035] Waiting for connection context
0811 19:01:05.300846 (+ 1528us) server_negotiation.cc:299] Negotiation successful
0811 19:01:05.301136 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 19:01:05.302347 22916 log.cc:1196] T 8797d08a7ebb48e6b93ea015e240b5bf P c5540dcd5d1846028c6ee159697fc086: Deleting WAL directory at /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-0/wal/wals/8797d08a7ebb48e6b93ea015e240b5bf
I20260811 19:01:05.302716 22453 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:01:05.306787 22918 ts_tablet_manager.cc:1918] T 8797d08a7ebb48e6b93ea015e240b5bf P 8b70c8cd4d2e4edea782997b0b2663aa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:01:05.307060 22918 log.cc:1196] T 8797d08a7ebb48e6b93ea015e240b5bf P 8b70c8cd4d2e4edea782997b0b2663aa: Deleting WAL directory at /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-2/wal/wals/8797d08a7ebb48e6b93ea015e240b5bf
I20260811 19:01:05.308300 22915 ts_tablet_manager.cc:1918] T 8797d08a7ebb48e6b93ea015e240b5bf P ed49ffb15359470f84f5fae8705c97e4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:01:05.308422 22918 ts_tablet_manager.cc:1939] T 8797d08a7ebb48e6b93ea015e240b5bf P 8b70c8cd4d2e4edea782997b0b2663aa: Deleting consensus metadata
I20260811 19:01:05.309785 22915 log.cc:1196] T 8797d08a7ebb48e6b93ea015e240b5bf P ed49ffb15359470f84f5fae8705c97e4: Deleting WAL directory at /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-1/wal/wals/8797d08a7ebb48e6b93ea015e240b5bf
I20260811 19:01:05.313405 22915 ts_tablet_manager.cc:1939] T 8797d08a7ebb48e6b93ea015e240b5bf P ed49ffb15359470f84f5fae8705c97e4: Deleting consensus metadata
I20260811 19:01:05.315435 22117 catalog_manager.cc:4841] TS 8b70c8cd4d2e4edea782997b0b2663aa (127.29.112.131:44959): tablet 8797d08a7ebb48e6b93ea015e240b5bf (table test-table [id=d5bafed92c414807b1d543932a1e0851]) successfully deleted
I20260811 19:01:05.316530 22116 catalog_manager.cc:4841] TS ed49ffb15359470f84f5fae8705c97e4 (127.29.112.130:46063): tablet 8797d08a7ebb48e6b93ea015e240b5bf (table test-table [id=d5bafed92c414807b1d543932a1e0851]) successfully deleted
I20260811 19:01:05.455168 22916 ts_tablet_manager.cc:1939] T 8797d08a7ebb48e6b93ea015e240b5bf P c5540dcd5d1846028c6ee159697fc086: Deleting consensus metadata
I20260811 19:01:05.456856 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 22206
I20260811 19:01:05.496588 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 22467
W20260811 19:01:05.496856 22116 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.112.129:41521 (error 108)
I20260811 19:01:05.499811 22921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:05.498716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:41521 (local address 127.0.0.1:40142)
0811 19:01:05.499382 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:05.499552 (+ 170us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.129:41521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":673,"threads_started":1}
W20260811 19:01:05.500198 22116 catalog_manager.cc:4584] TS c5540dcd5d1846028c6ee159697fc086 (127.29.112.129:41521): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8797d08a7ebb48e6b93ea015e240b5bf: Network error: Client connection negotiation failed: client connection to 127.29.112.129:41521: connect: Connection refused (error 111)
I20260811 19:01:05.518930 22871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:05.498188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56198 (local address 127.29.112.130:46063)
0811 19:01:05.498383 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0811 19:01:05.498388 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:01:05.498410 (+ 22us) server_negotiation.cc:407] Connection header received
0811 19:01:05.498471 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:01:05.498474 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:01:05.498591 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:01:05.498726 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:01:05.500739 (+ 2013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:05.501561 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:05.504287 (+ 2726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:05.504991 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:05.505151 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:05.510176 (+ 5025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:01:05.510197 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:01:05.510217 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:01:05.510277 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:01:05.515101 (+ 4824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:01:05.516151 (+ 1050us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:01:05.516164 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:01:05.516169 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:01:05.516365 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:01:05.516912 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:01:05.516917 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:01:05.516923 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:01:05.517469 (+ 546us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:01:05.517798 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0811 19:01:05.518202 (+ 404us) server_negotiation.cc:299] Negotiation successful
0811 19:01:05.518469 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 19:01:05.520370 22635 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:01:05.555449 22921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:05.554558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:41521 (local address 127.0.0.1:40144)
0811 19:01:05.555077 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:05.555202 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.129:41521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260811 19:01:05.556748 22921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:05.556332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:41521 (local address 127.0.0.1:40158)
0811 19:01:05.556499 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:05.556585 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.129:41521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260811 19:01:05.617841 22921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:05.616978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:41521 (local address 127.0.0.1:40164)
0811 19:01:05.617372 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:05.617543 (+ 171us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.129:41521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158}
I20260811 19:01:05.619608 22921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:05.618820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:41521 (local address 127.0.0.1:40172)
0811 19:01:05.619204 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:05.619350 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.129:41521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":246,"threads_started":1}
I20260811 19:01:05.642098 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 22467
I20260811 19:01:05.679701 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 22647
I20260811 19:01:05.701462 22926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:05.700916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:41521 (local address 127.0.0.1:40174)
0811 19:01:05.701156 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:05.701265 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.129:41521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260811 19:01:05.702720 22926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:05.702236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:41521 (local address 127.0.0.1:40176)
0811 19:01:05.702430 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:05.702568 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.129:41521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260811 19:01:05.706868 22868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:05.681431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38934 (local address 127.29.112.131:44959)
0811 19:01:05.681824 (+ 393us) server_negotiation.cc:206] Beginning negotiation
0811 19:01:05.681837 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 19:01:05.681997 (+ 160us) server_negotiation.cc:407] Connection header received
0811 19:01:05.682200 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:01:05.682207 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:01:05.682336 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:01:05.682507 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:01:05.684522 (+ 2015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:05.686244 (+ 1722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:05.688856 (+ 2612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:05.689359 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:05.689542 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:05.693700 (+ 4158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:01:05.693715 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:01:05.693732 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:01:05.693771 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:01:05.700069 (+ 6298us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:01:05.702475 (+ 2406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:01:05.702489 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:01:05.702492 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:01:05.702728 (+ 236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:01:05.703359 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:01:05.703363 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:01:05.703365 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:01:05.703803 (+ 438us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:01:05.704357 (+ 554us) server_negotiation.cc:1035] Waiting for connection context
0811 19:01:05.706281 (+ 1924us) server_negotiation.cc:299] Negotiation successful
0811 19:01:05.706487 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":218}
I20260811 19:01:05.707849 22832 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:01:05.863006 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 22647
I20260811 19:01:05.875784 22926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:05.874997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:41521 (local address 127.0.0.1:40192)
0811 19:01:05.875234 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:05.875504 (+ 270us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.129:41521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260811 19:01:05.877424 22926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:05.876957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:41521 (local address 127.0.0.1:40208)
0811 19:01:05.877157 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:05.877267 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.129:41521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260811 19:01:05.898510 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 22084
I20260811 19:01:05.899904 22189 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:01:06.036031 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 22084
I20260811 19:01:06.065470 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.190:37101
--webserver_interface=127.29.112.190
--webserver_port=32779
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab
--principal=kudu/127.29.112.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:41831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.190:37101
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:01:06.564738 22930 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:01:06.565426 22930 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:01:06.565732 22930 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:01:06.577042 22930 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:01:06.577291 22930 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:01:06.577514 22930 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:01:06.578008 22930 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:01:06.578150 22930 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:01:06.591156 22930 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:41831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.112.190:37101
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab
--principal=kudu/127.29.112.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.190:37101
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.112.190
--webserver_port=32779
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:01:06.592818 22930 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:01:06.594310 22930 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:01:06.616453 22935 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:01:06.616384 22938 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:01:06.616367 22941 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:01:06.618657 22930 server_base.cc:1034] running on GCE node
I20260811 19:01:06.619936 22930 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:01:06.622915 22930 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:01:06.624466 22930 hybrid_clock.cc:648] HybridClock initialized: now 1786474866624332 us; error 44 us; skew 500 ppm
I20260811 19:01:06.625514 22930 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:01:06 dist-test-slave-h6n0 krb5kdc[21166](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474866, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:01:06.635701 22930 init.cc:377] Logged in from keytab as kudu/127.29.112.190@KRBTEST.COM (short username kudu)
I20260811 19:01:06.638094 22930 webserver.cc:460] Webserver started at http://127.29.112.190:32779/ using document root <none> and password file <none>
I20260811 19:01:06.639086 22930 fs_manager.cc:359] Metadata directory not provided
I20260811 19:01:06.639225 22930 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:01:06.647580 22930 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.006s sys 0.000s
I20260811 19:01:06.653951 22958 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:01:06.659528 22930 fs_manager.cc:725] Time spent opening block manager: real 0.009s user 0.009s sys 0.000s
I20260811 19:01:06.660432 22930 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/master-0/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/master-0/wal
uuid: "4985530a07eb4edeaa3cae40b71554d8"
format_stamp: "Formatted at 2026-08-11 19:00:58 on dist-test-slave-h6n0"
server_key: "664f1286906a295f39575e04fc41b0b6"
server_key_iv: "56d23316f6394ca240d593c3de306ea9"
server_key_version: "encryptionkey@0"
I20260811 19:01:06.663684 22930 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:01:06.713707 22930 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:01:06.721534 22930 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:01:06.721899 22930 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:01:06.758387 22930 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.190:37101
I20260811 19:01:06.758631 23034 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.190:37101 every 8 connection(s)
I20260811 19:01:06.760375 22930 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
I20260811 19:01:06.765653 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 22930
I20260811 19:01:06.766870 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.129:41521
--local_ip_for_outbound_sockets=127.29.112.129
--tserver_master_addrs=127.29.112.190:37101
--webserver_port=37943
--webserver_interface=127.29.112.129
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.129.keytab
--principal=kudu/127.29.112.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:41831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 19:01:06.772783 23035 sys_catalog.cc:263] Verifying existing consensus state
I20260811 19:01:06.777884 23035 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4985530a07eb4edeaa3cae40b71554d8: Bootstrap starting.
I20260811 19:01:06.823398 23035 log.cc:824] T 00000000000000000000000000000000 P 4985530a07eb4edeaa3cae40b71554d8: Log is configured to *not* fsync() on all Append() calls
I20260811 19:01:06.860231 23035 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4985530a07eb4edeaa3cae40b71554d8: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 19:01:06.861130 23035 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4985530a07eb4edeaa3cae40b71554d8: Bootstrap complete.
I20260811 19:01:06.882341 23035 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4985530a07eb4edeaa3cae40b71554d8 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4985530a07eb4edeaa3cae40b71554d8" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 37101 } }
I20260811 19:01:06.884466 23035 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4985530a07eb4edeaa3cae40b71554d8 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4985530a07eb4edeaa3cae40b71554d8, State: Initialized, Role: FOLLOWER
I20260811 19:01:06.885613 23035 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4985530a07eb4edeaa3cae40b71554d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4985530a07eb4edeaa3cae40b71554d8" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 37101 } }
I20260811 19:01:06.886062 23035 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4985530a07eb4edeaa3cae40b71554d8 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:01:06.886252 23035 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4985530a07eb4edeaa3cae40b71554d8 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:01:06.886931 23035 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4985530a07eb4edeaa3cae40b71554d8 [term 2 FOLLOWER]: Advancing to term 3
I20260811 19:01:06.897174 23035 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4985530a07eb4edeaa3cae40b71554d8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4985530a07eb4edeaa3cae40b71554d8" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 37101 } }
I20260811 19:01:06.898030 23035 leader_election.cc:302] T 00000000000000000000000000000000 P 4985530a07eb4edeaa3cae40b71554d8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4985530a07eb4edeaa3cae40b71554d8; no voters:
I20260811 19:01:06.898989 23035 leader_election.cc:288] T 00000000000000000000000000000000 P 4985530a07eb4edeaa3cae40b71554d8 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 19:01:06.899160 23042 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4985530a07eb4edeaa3cae40b71554d8 [term 3 FOLLOWER]: Leader election won for term 3
I20260811 19:01:06.900847 23042 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4985530a07eb4edeaa3cae40b71554d8 [term 3 LEADER]: Becoming Leader. State: Replica: 4985530a07eb4edeaa3cae40b71554d8, State: Running, Role: LEADER
I20260811 19:01:06.901564 23042 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4985530a07eb4edeaa3cae40b71554d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4985530a07eb4edeaa3cae40b71554d8" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 37101 } }
I20260811 19:01:06.901952 23035 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4985530a07eb4edeaa3cae40b71554d8 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:01:06.905876 23044 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4985530a07eb4edeaa3cae40b71554d8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4985530a07eb4edeaa3cae40b71554d8. Latest consensus state: current_term: 3 leader_uuid: "4985530a07eb4edeaa3cae40b71554d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4985530a07eb4edeaa3cae40b71554d8" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 37101 } } }
I20260811 19:01:06.905879 23043 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4985530a07eb4edeaa3cae40b71554d8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "4985530a07eb4edeaa3cae40b71554d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4985530a07eb4edeaa3cae40b71554d8" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 37101 } } }
I20260811 19:01:06.907353 23043 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4985530a07eb4edeaa3cae40b71554d8 [sys.catalog]: This master's current role is: LEADER
I20260811 19:01:06.907354 23044 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4985530a07eb4edeaa3cae40b71554d8 [sys.catalog]: This master's current role is: LEADER
I20260811 19:01:06.917140 23050 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:01:06.929899 23050 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:01:06.931716 23050 catalog_manager.cc:1259] Loaded cluster ID: 3cfa2eb09efe418f8e8c0c1db4d80d41
I20260811 19:01:06.931845 23050 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260811 19:01:06.935117 23050 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fdb959059ad google::LogMessage::Fail() at ??:0
@ 0x7fdb9590888a google::LogMessage::SendToLog() at ??:0
@ 0x7fdb959054de google::LogMessage::Flush() at ??:0
@ 0x7fdb95906a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fdbab338512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7fdbab335976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fdbab3bd610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7fdbab3bd439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fdb98551345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fdb9855491c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7fdb98554799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fdb9852feaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7fdb9b8096db start_thread at ??:0
@ 0x7fdb9479f61f clone at ??:0
W20260811 19:01:07.392215 23039 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:01:07.392591 23039 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:01:07.392766 23039 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:01:07.401650 23039 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:01:07.401859 23039 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:01:07.402060 23039 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.129
I20260811 19:01:07.418392 23039 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:41831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.129.keytab
--principal=kudu/127.29.112.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.129:41521
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.112.129
--webserver_port=37943
--tserver_master_addrs=127.29.112.190:37101
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.129
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:01:07.420013 23039 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:01:07.421573 23039 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:01:07.437683 23097 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:01:07.437817 23096 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:01:07.437817 23101 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:01:07.439014 23039 server_base.cc:1034] running on GCE node
I20260811 19:01:07.439776 23039 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:01:07.440966 23039 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:01:07.442272 23039 hybrid_clock.cc:648] HybridClock initialized: now 1786474867442181 us; error 79 us; skew 500 ppm
I20260811 19:01:07.442793 23039 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:01:07 dist-test-slave-h6n0 krb5kdc[21166](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474867, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:01:07.447310 23039 init.cc:377] Logged in from keytab as kudu/127.29.112.129@KRBTEST.COM (short username kudu)
I20260811 19:01:07.449564 23039 webserver.cc:460] Webserver started at http://127.29.112.129:37943/ using document root <none> and password file <none>
I20260811 19:01:07.450413 23039 fs_manager.cc:359] Metadata directory not provided
I20260811 19:01:07.450590 23039 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:01:07.458729 23039 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 19:01:07.462186 23114 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:01:07.463910 23039 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 19:01:07.464093 23039 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-0/wal
uuid: "c5540dcd5d1846028c6ee159697fc086"
format_stamp: "Formatted at 2026-08-11 19:00:59 on dist-test-slave-h6n0"
server_key: "e368153fff9f1dc9903d82a309663ea9"
server_key_iv: "0ff1c8300acc5725394f2b0c52a33c6c"
server_key_version: "encryptionkey@0"
I20260811 19:01:07.465201 23039 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:01:07.483949 23039 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:01:07.491426 23039 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:01:07.492250 23039 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:01:07.494372 23039 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:01:07.496264 23039 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:01:07.496381 23039 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:01:07.496522 23039 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:01:07.496613 23039 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:01:07.545804 23039 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.129:41521
I20260811 19:01:07.545943 23300 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.129:41521 every 8 connection(s)
I20260811 19:01:07.548454 23039 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-0/data/info.pb
I20260811 19:01:07.554982 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 23039
I20260811 19:01:07.556008 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.130:46063
--local_ip_for_outbound_sockets=127.29.112.130
--tserver_master_addrs=127.29.112.190:37101
--webserver_port=40137
--webserver_interface=127.29.112.130
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.130.keytab
--principal=kudu/127.29.112.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:41831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 19:01:07.556846 23304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:07.553704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37101 (local address 127.29.112.129:38063)
0811 19:01:07.554932 (+ 1228us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:07.555304 (+ 372us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:37101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":737,"thread_start_us":455,"threads_started":1}
W20260811 19:01:07.557909 23301 heartbeater.cc:644] Failed to heartbeat to 127.29.112.190:37101 (0 consecutive failures): Network error: Failed to ping master at 127.29.112.190:37101: Client connection negotiation failed: client connection to 127.29.112.190:37101: connect: Connection refused (error 111)
I20260811 19:01:07.559273 23304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:07.558795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37101 (local address 127.29.112.129:53843)
0811 19:01:07.559043 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:07.559137 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:37101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120}
I20260811 19:01:07.560398 23304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:07.560017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37101 (local address 127.29.112.129:47813)
0811 19:01:07.560189 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:07.560279 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:37101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
W20260811 19:01:07.560767 23301 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 19:01:08.004630 23305 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:01:08.005240 23305 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:01:08.005440 23305 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:01:08.014117 23305 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:01:08.014317 23305 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:01:08.014546 23305 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.130
I20260811 19:01:08.027017 23305 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:41831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.130.keytab
--principal=kudu/127.29.112.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.130:46063
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.112.130
--webserver_port=40137
--tserver_master_addrs=127.29.112.190:37101
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.130
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:01:08.028882 23305 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:01:08.031019 23305 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:01:08.056197 23311 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:01:08.056598 23318 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:01:08.056655 23313 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:01:08.059273 23305 server_base.cc:1034] running on GCE node
I20260811 19:01:08.060124 23305 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:01:08.061424 23305 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:01:08.062898 23305 hybrid_clock.cc:648] HybridClock initialized: now 1786474868062777 us; error 69 us; skew 500 ppm
I20260811 19:01:08.063571 23305 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:01:08 dist-test-slave-h6n0 krb5kdc[21166](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474868, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:01:08.067842 23305 init.cc:377] Logged in from keytab as kudu/127.29.112.130@KRBTEST.COM (short username kudu)
I20260811 19:01:08.070600 23305 webserver.cc:460] Webserver started at http://127.29.112.130:40137/ using document root <none> and password file <none>
I20260811 19:01:08.071617 23305 fs_manager.cc:359] Metadata directory not provided
I20260811 19:01:08.071776 23305 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:01:08.081326 23305 fs_manager.cc:709] Time spent opening directory manager: real 0.007s user 0.003s sys 0.006s
I20260811 19:01:08.085362 23329 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:01:08.088613 23305 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.002s sys 0.000s
I20260811 19:01:08.088814 23305 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-1/wal
uuid: "ed49ffb15359470f84f5fae8705c97e4"
format_stamp: "Formatted at 2026-08-11 19:01:00 on dist-test-slave-h6n0"
server_key: "084c999a79f65c0a1b1a0cdf6a124b49"
server_key_iv: "7c1812b0d2ac818c81e65cb084c32743"
server_key_version: "encryptionkey@0"
I20260811 19:01:08.089972 23305 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:01:08.118557 23305 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:01:08.125973 23305 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:01:08.126246 23305 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:01:08.127416 23305 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:01:08.128921 23305 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:01:08.129032 23305 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:01:08.129106 23305 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:01:08.129163 23305 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:01:08.176954 23305 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.130:46063
I20260811 19:01:08.177021 23501 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.130:46063 every 8 connection(s)
I20260811 19:01:08.178514 23305 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-1/data/info.pb
I20260811 19:01:08.179854 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 23305
I20260811 19:01:08.180595 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.131:44959
--local_ip_for_outbound_sockets=127.29.112.131
--tserver_master_addrs=127.29.112.190:37101
--webserver_port=38097
--webserver_interface=127.29.112.131
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.131.keytab
--principal=kudu/127.29.112.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:41831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 19:01:08.186018 23504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:08.182999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37101 (local address 127.29.112.130:42007)
0811 19:01:08.184122 (+ 1123us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:08.184586 (+ 464us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:37101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":752,"thread_start_us":458,"threads_started":1}
W20260811 19:01:08.186985 23502 heartbeater.cc:644] Failed to heartbeat to 127.29.112.190:37101 (0 consecutive failures): Network error: Failed to ping master at 127.29.112.190:37101: Client connection negotiation failed: client connection to 127.29.112.190:37101: connect: Connection refused (error 111)
I20260811 19:01:08.188508 23504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:08.187959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37101 (local address 127.29.112.130:49383)
0811 19:01:08.188205 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:08.188329 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:37101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260811 19:01:08.190127 23504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:08.189462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37101 (local address 127.29.112.130:44349)
0811 19:01:08.189720 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:08.189960 (+ 240us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:37101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116}
W20260811 19:01:08.190768 23502 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 19:01:08.562906 23508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:08.561658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37101 (local address 127.29.112.129:47283)
0811 19:01:08.562258 (+ 600us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:08.562407 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:37101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":316,"threads_started":1}
W20260811 19:01:08.676826 23507 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:01:08.677536 23507 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:01:08.677767 23507 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:01:08.687347 23507 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:01:08.687568 23507 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:01:08.687726 23507 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.131
I20260811 19:01:08.703224 23507 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:41831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.131.keytab
--principal=kudu/127.29.112.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.131:44959
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.112.131
--webserver_port=38097
--tserver_master_addrs=127.29.112.190:37101
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.131
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:01:08.704691 23507 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:01:08.706378 23507 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:01:08.724673 23518 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:01:08.725201 23521 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:01:08.725656 23519 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:01:08.728179 23507 server_base.cc:1034] running on GCE node
I20260811 19:01:08.729007 23507 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:01:08.730131 23507 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:01:08.733808 23507 hybrid_clock.cc:648] HybridClock initialized: now 1786474868733569 us; error 71 us; skew 500 ppm
I20260811 19:01:08.734315 23507 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:01:08 dist-test-slave-h6n0 krb5kdc[21166](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474868, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:01:08.739794 23507 init.cc:377] Logged in from keytab as kudu/127.29.112.131@KRBTEST.COM (short username kudu)
I20260811 19:01:08.741757 23507 webserver.cc:460] Webserver started at http://127.29.112.131:38097/ using document root <none> and password file <none>
I20260811 19:01:08.742810 23507 fs_manager.cc:359] Metadata directory not provided
I20260811 19:01:08.743064 23507 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:01:08.752066 23507 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.006s sys 0.001s
I20260811 19:01:08.757754 23530 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:01:08.760380 23507 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.001s
I20260811 19:01:08.760615 23507 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-2/wal
uuid: "8b70c8cd4d2e4edea782997b0b2663aa"
format_stamp: "Formatted at 2026-08-11 19:01:00 on dist-test-slave-h6n0"
server_key: "17151f16d18aa4af7c494b1266623e68"
server_key_iv: "e7d77d4a59c2b5dcd2cca775b8306f50"
server_key_version: "encryptionkey@0"
I20260811 19:01:08.761799 23507 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:01:08.781229 23507 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:01:08.786412 23507 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:01:08.786990 23507 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:01:08.788544 23507 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:01:08.790191 23507 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:01:08.790297 23507 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:01:08.790393 23507 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:01:08.790448 23507 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:01:08.845070 23507 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.131:44959
I20260811 19:01:08.845172 23711 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.131:44959 every 8 connection(s)
I20260811 19:01:08.847012 23507 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474615205622-30146-0/minicluster-data/ts-2/data/info.pb
I20260811 19:01:08.855216 23714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:08.851843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37101 (local address 127.29.112.131:55341)
0811 19:01:08.853068 (+ 1225us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:08.853513 (+ 445us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:37101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":844,"thread_start_us":571,"threads_started":1}
I20260811 19:01:08.855949 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 23507
W20260811 19:01:08.856494 23712 heartbeater.cc:644] Failed to heartbeat to 127.29.112.190:37101 (0 consecutive failures): Network error: Failed to ping master at 127.29.112.190:37101: Client connection negotiation failed: client connection to 127.29.112.190:37101: connect: Connection refused (error 111)
I20260811 19:01:08.857977 23714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:08.857475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37101 (local address 127.29.112.131:34075)
0811 19:01:08.857739 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:08.857835 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:37101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260811 19:01:08.859189 23714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:08.858798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37101 (local address 127.29.112.131:39429)
0811 19:01:08.858966 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:08.859067 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:37101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260811 19:01:08.859444 23712 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 19:01:09.193403 23718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:09.191944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37101 (local address 127.29.112.130:44485)
0811 19:01:09.192907 (+ 963us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:09.193106 (+ 199us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:37101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":827,"thread_start_us":643,"threads_started":1}
I20260811 19:01:09.564872 23719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:09.564087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37101 (local address 127.29.112.129:50665)
0811 19:01:09.564592 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:09.564692 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:37101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":255,"threads_started":1}
I20260811 19:01:09.861371 23720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:09.860156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37101 (local address 127.29.112.131:37509)
0811 19:01:09.860934 (+ 778us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:09.861110 (+ 176us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:37101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":618,"thread_start_us":506,"threads_started":1}
I20260811 19:01:10.196311 23721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:10.195299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37101 (local address 127.29.112.130:39827)
0811 19:01:10.196009 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:10.196112 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:37101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":578,"thread_start_us":387,"threads_started":1}
I20260811 19:01:10.567241 23723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:10.566286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37101 (local address 127.29.112.129:44629)
0811 19:01:10.566840 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:10.566941 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:37101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":321,"threads_started":1}
I20260811 19:01:10.863502 23726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:10.862689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37101 (local address 127.29.112.131:46607)
0811 19:01:10.863206 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:10.863297 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:37101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":268,"threads_started":1}
I20260811 19:01:11.198729 23727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:11.197311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37101 (local address 127.29.112.130:51479)
0811 19:01:11.198058 (+ 747us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:11.198440 (+ 382us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:37101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":595,"thread_start_us":379,"threads_started":1}
I20260811 19:01:11.570233 23728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:11.568760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37101 (local address 127.29.112.129:53085)
0811 19:01:11.569836 (+ 1076us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:11.569982 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:37101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":921,"thread_start_us":559,"threads_started":1}
I20260811 19:01:11.865653 23729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:11.864677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37101 (local address 127.29.112.131:36125)
0811 19:01:11.865301 (+ 624us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:11.865411 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:37101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":305,"threads_started":1}
I20260811 19:01:12.201047 23730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:12.200116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37101 (local address 127.29.112.130:52001)
0811 19:01:12.200724 (+ 608us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:12.200819 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:37101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":331,"threads_started":1}
I20260811 19:01:12.573455 23731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:12.571915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37101 (local address 127.29.112.129:42503)
0811 19:01:12.572876 (+ 961us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:12.573000 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:37101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":818,"thread_start_us":590,"threads_started":1}
I20260811 19:01:12.868502 23732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:12.867193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37101 (local address 127.29.112.131:48761)
0811 19:01:12.868007 (+ 814us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:12.868124 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:37101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":707,"thread_start_us":420,"threads_started":1}
I20260811 19:01:13.203639 23733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:13.202269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37101 (local address 127.29.112.130:53745)
0811 19:01:13.203132 (+ 863us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:13.203393 (+ 261us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:37101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":707,"thread_start_us":479,"threads_started":1}
I20260811 19:01:13.576201 23735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:13.575225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37101 (local address 127.29.112.129:37681)
0811 19:01:13.575874 (+ 649us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:13.576001 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:37101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":368,"threads_started":1}
I20260811 19:01:13.871436 23737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:13.870097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37101 (local address 127.29.112.131:42417)
0811 19:01:13.870936 (+ 839us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:13.871059 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:37101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":677,"thread_start_us":501,"threads_started":1}
I20260811 19:01:14.206127 23738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:14.205090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37101 (local address 127.29.112.130:54891)
0811 19:01:14.205759 (+ 669us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:14.205885 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:37101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":363,"threads_started":1}
I20260811 19:01:14.578843 23739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:14.577782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37101 (local address 127.29.112.129:57305)
0811 19:01:14.578413 (+ 631us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:14.578510 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:37101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":500,"thread_start_us":307,"threads_started":1}
I20260811 19:01:14.873545 23740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:14.872740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37101 (local address 127.29.112.131:56765)
0811 19:01:14.873250 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:14.873357 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:37101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":256,"threads_started":1}
I20260811 19:01:15.208730 23741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:15.207628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37101 (local address 127.29.112.130:50233)
0811 19:01:15.208362 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:15.208489 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:37101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":560,"thread_start_us":384,"threads_started":1}
I20260811 19:01:15.580996 23742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:15.579960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37101 (local address 127.29.112.129:47527)
0811 19:01:15.580660 (+ 700us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:15.580763 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:37101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":585,"thread_start_us":439,"threads_started":1}
I20260811 19:01:15.876055 23743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:15.874854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37101 (local address 127.29.112.131:52631)
0811 19:01:15.875556 (+ 702us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:15.875744 (+ 188us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:37101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":552,"thread_start_us":384,"threads_started":1}
I20260811 19:01:16.211230 23744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:16.210268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37101 (local address 127.29.112.130:57621)
0811 19:01:16.210929 (+ 661us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:16.211042 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:37101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":547,"thread_start_us":409,"threads_started":1}
I20260811 19:01:16.583137 23746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:16.582146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37101 (local address 127.29.112.129:53725)
0811 19:01:16.582808 (+ 662us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:16.582924 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:37101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":547,"thread_start_us":328,"threads_started":1}
I20260811 19:01:16.878018 23748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:16.877241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37101 (local address 127.29.112.131:37513)
0811 19:01:16.877752 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:16.877842 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:37101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":283,"threads_started":1}
I20260811 19:01:17.213840 23749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:17.212930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37101 (local address 127.29.112.130:51771)
0811 19:01:17.213497 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:17.213634 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:37101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":262,"threads_started":1}
I20260811 19:01:17.585057 23750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:17.584218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37101 (local address 127.29.112.129:36293)
0811 19:01:17.584795 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:17.584887 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:37101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":355,"threads_started":1}
I20260811 19:01:17.880406 23751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:17.879358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37101 (local address 127.29.112.131:37789)
0811 19:01:17.880110 (+ 752us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:17.880212 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:37101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":601,"thread_start_us":425,"threads_started":1}
I20260811 19:01:18.216285 23753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:18.215351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37101 (local address 127.29.112.130:41797)
0811 19:01:18.215988 (+ 637us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:18.216097 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:37101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":320,"threads_started":1}
I20260811 19:01:18.587546 23755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:18.586476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37101 (local address 127.29.112.129:56009)
0811 19:01:18.587185 (+ 709us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:18.587300 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:37101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":594,"thread_start_us":375,"threads_started":1}
I20260811 19:01:18.882858 23756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:18.881921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37101 (local address 127.29.112.131:42045)
0811 19:01:18.882441 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:18.882669 (+ 228us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:37101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":288,"threads_started":1}
I20260811 19:01:19.219309 23759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:19.218061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37101 (local address 127.29.112.130:45801)
0811 19:01:19.218984 (+ 923us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:19.219103 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:37101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":586,"thread_start_us":364,"threads_started":1}
I20260811 19:01:19.590413 23761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:19.589084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37101 (local address 127.29.112.129:45921)
0811 19:01:19.589907 (+ 823us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:19.590025 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:37101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":678,"thread_start_us":531,"threads_started":1}
I20260811 19:01:19.885488 23763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:19.884211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37101 (local address 127.29.112.131:55193)
0811 19:01:19.884937 (+ 726us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:19.885101 (+ 164us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:37101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":360,"threads_started":1}
I20260811 19:01:20.221757 23764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:20.220836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37101 (local address 127.29.112.130:38507)
0811 19:01:20.221461 (+ 625us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:20.221566 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:37101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":380,"threads_started":1}
I20260811 19:01:20.593525 23765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:20.591960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37101 (local address 127.29.112.129:44705)
0811 19:01:20.592859 (+ 899us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:20.592991 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:37101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":760,"thread_start_us":464,"threads_started":1}
I20260811 19:01:20.887753 23766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:20.886679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37101 (local address 127.29.112.131:59823)
0811 19:01:20.887364 (+ 685us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:20.887507 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:37101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":332,"threads_started":1}
I20260811 19:01:21.224243 23767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:21.223259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37101 (local address 127.29.112.130:50259)
0811 19:01:21.223893 (+ 634us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:21.224024 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:37101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":348,"threads_started":1}
I20260811 19:01:21.596565 23768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:21.595113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37101 (local address 127.29.112.129:48643)
0811 19:01:21.596095 (+ 982us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:21.596308 (+ 213us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:37101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":702,"thread_start_us":522,"threads_started":1}
I20260811 19:01:21.890072 23769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:21.889104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37101 (local address 127.29.112.131:35309)
0811 19:01:21.889706 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:21.889815 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:37101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":322,"threads_started":1}
I20260811 19:01:22.227031 23770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:22.225515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37101 (local address 127.29.112.130:48515)
0811 19:01:22.226319 (+ 804us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:22.226461 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:37101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":640,"thread_start_us":516,"threads_started":1}
I20260811 19:01:22.599785 23771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:22.598448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37101 (local address 127.29.112.129:44665)
0811 19:01:22.599402 (+ 954us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:22.599557 (+ 155us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:37101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":663,"thread_start_us":549,"threads_started":1}
I20260811 19:01:22.892729 23772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:22.891655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37101 (local address 127.29.112.131:60961)
0811 19:01:22.892327 (+ 672us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:22.892450 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:37101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":356,"threads_started":1}
I20260811 19:01:23.229301 23773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:23.228456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37101 (local address 127.29.112.130:46383)
0811 19:01:23.228998 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:23.229111 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:37101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":323,"threads_started":1}
I20260811 19:01:23.602208 23775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:23.601327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37101 (local address 127.29.112.129:36621)
0811 19:01:23.601911 (+ 584us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:23.602029 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:37101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":318,"threads_started":1}
I20260811 19:01:23.862634 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 23039
I20260811 19:01:23.887786 23776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:23.864114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46218 (local address 127.29.112.129:41521)
0811 19:01:23.864797 (+ 683us) server_negotiation.cc:206] Beginning negotiation
0811 19:01:23.864817 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 19:01:23.864884 (+ 67us) server_negotiation.cc:407] Connection header received
0811 19:01:23.867683 (+ 2799us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:01:23.867693 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:01:23.868168 (+ 475us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:01:23.868551 (+ 383us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:01:23.870387 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:23.872743 (+ 2356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:23.875014 (+ 2271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:23.875889 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:23.876286 (+ 397us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:23.879942 (+ 3656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:01:23.879952 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:01:23.879970 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:01:23.880014 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:01:23.883758 (+ 3744us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:01:23.884595 (+ 837us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:01:23.884608 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:01:23.884615 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:01:23.885035 (+ 420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:01:23.885533 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:01:23.885537 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:01:23.885539 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:01:23.886135 (+ 596us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:01:23.886492 (+ 357us) server_negotiation.cc:1035] Waiting for connection context
0811 19:01:23.886983 (+ 491us) server_negotiation.cc:299] Negotiation successful
0811 19:01:23.887469 (+ 486us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":481,"thread_start_us":312,"threads_started":1}
I20260811 19:01:23.889258 23291 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:01:23.895475 23780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:23.894148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:37101 (local address 127.29.112.131:51307)
0811 19:01:23.895007 (+ 859us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:23.895164 (+ 157us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:37101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":679,"thread_start_us":495,"threads_started":1}
I20260811 19:01:23.981061 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 23039
I20260811 19:01:24.012825 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 23305
I20260811 19:01:24.035255 23781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:24.014146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52714 (local address 127.29.112.130:46063)
0811 19:01:24.014849 (+ 703us) server_negotiation.cc:206] Beginning negotiation
0811 19:01:24.014864 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 19:01:24.014903 (+ 39us) server_negotiation.cc:407] Connection header received
0811 19:01:24.016511 (+ 1608us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:01:24.016521 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:01:24.016960 (+ 439us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:01:24.017344 (+ 384us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:01:24.019142 (+ 1798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:24.020666 (+ 1524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:24.022132 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:24.022936 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:24.023140 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:24.027316 (+ 4176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:01:24.027325 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:01:24.027344 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:01:24.027395 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:01:24.030995 (+ 3600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:01:24.031774 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:01:24.031781 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:01:24.031788 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:01:24.031973 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:01:24.032676 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:01:24.032683 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:01:24.032686 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:01:24.033294 (+ 608us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:01:24.033625 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0811 19:01:24.034176 (+ 551us) server_negotiation.cc:299] Negotiation successful
0811 19:01:24.034723 (+ 547us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":533,"thread_start_us":305,"threads_started":1}
I20260811 19:01:24.037200 23496 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:01:24.128077 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 23305
I20260811 19:01:24.159044 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 23507
I20260811 19:01:24.179103 23783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:24.160050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48372 (local address 127.29.112.131:44959)
0811 19:01:24.160629 (+ 579us) server_negotiation.cc:206] Beginning negotiation
0811 19:01:24.160644 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 19:01:24.160687 (+ 43us) server_negotiation.cc:407] Connection header received
0811 19:01:24.162631 (+ 1944us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:01:24.162641 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:01:24.163075 (+ 434us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:01:24.163426 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:01:24.165131 (+ 1705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:24.166651 (+ 1520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:24.168190 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:24.168899 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:24.169103 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:24.172181 (+ 3078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:01:24.172188 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:01:24.172204 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:01:24.172251 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:01:24.175448 (+ 3197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:01:24.176176 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:01:24.176184 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:01:24.176191 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:01:24.176364 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:01:24.176778 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:01:24.176783 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:01:24.176788 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:01:24.177344 (+ 556us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:01:24.177672 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0811 19:01:24.178159 (+ 487us) server_negotiation.cc:299] Negotiation successful
0811 19:01:24.178671 (+ 512us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":413,"thread_start_us":255,"threads_started":1}
I20260811 19:01:24.180546 23706 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:01:24.275900 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 23507
2026-08-11T19:01:24Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (28280 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:01:24 dist-test-slave-h6n0 krb5kdc[23787](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:01:24 dist-test-slave-h6n0 krb5kdc[23787](info): set up 2 sockets
Aug 11 19:01:24 dist-test-slave-h6n0 krb5kdc[23787](info): commencing operation
krb5kdc: starting...
W20260811 19:01:26.018217 30146 mini_kdc.cc:121] Time spent starting KDC: real 1.670s user 0.005s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/krb5kdc/test-user.keytab.
Aug 11 19:01:26 dist-test-slave-h6n0 krb5kdc[23787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474886, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:01:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:01:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.112.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab.
Entry for principal kudu/127.29.112.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab.
W20260811 19:01:26.528196 30146 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.29.112.190: real 0.103s user 0.001s sys 0.038s
I20260811 19:01:26.530987 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.190:45339
--webserver_interface=127.29.112.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab
--principal=kudu/127.29.112.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:36075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.190:45339
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:01:27.166576 23808 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:01:27.167109 23808 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:01:27.167413 23808 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:01:27.182404 23808 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 19:01:27.182638 23808 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 19:01:27.182704 23808 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 19:01:27.182803 23808 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:01:27.182905 23808 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:01:27.183049 23808 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:01:27.183116 23808 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:01:27.183195 23808 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:01:27.202009 23808 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:36075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.29.112.190:45339
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab
--principal=kudu/127.29.112.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.190:45339
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.112.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:01:27.205118 23808 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:01:27.208494 23808 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:01:27.236479 23818 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:01:27.236575 23817 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:01:27.236377 23821 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:01:27.238253 23808 server_base.cc:1034] running on GCE node
I20260811 19:01:27.239689 23808 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:01:27.241775 23808 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:01:27.243135 23808 hybrid_clock.cc:648] HybridClock initialized: now 1786474887243035 us; error 31 us; skew 500 ppm
I20260811 19:01:27.243732 23808 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:01:27 dist-test-slave-h6n0 krb5kdc[23787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474887, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:01:27.250353 23808 init.cc:377] Logged in from keytab as kudu/127.29.112.190@KRBTEST.COM (short username kudu)
I20260811 19:01:27.254115 23808 webserver.cc:460] Webserver started at http://127.29.112.190:43159/ using document root <none> and password file <none>
I20260811 19:01:27.255316 23808 fs_manager.cc:359] Metadata directory not provided
I20260811 19:01:27.255489 23808 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:01:27.255960 23808 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:01:27.262626 23808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/master-0/data/instance:
uuid: "394f0e9f175e4f24b7e1081aaeb6fb3b"
format_stamp: "Formatted at 2026-08-11 19:01:27 on dist-test-slave-h6n0"
server_key: "54b1f8faf9111f2625fb7e8489539574"
server_key_iv: "69b659430a273751b5130371a7442b26"
server_key_version: "encryptionkey@0"
I20260811 19:01:27.263867 23808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/master-0/wal/instance:
uuid: "394f0e9f175e4f24b7e1081aaeb6fb3b"
format_stamp: "Formatted at 2026-08-11 19:01:27 on dist-test-slave-h6n0"
server_key: "54b1f8faf9111f2625fb7e8489539574"
server_key_iv: "69b659430a273751b5130371a7442b26"
server_key_version: "encryptionkey@0"
I20260811 19:01:27.275473 23808 fs_manager.cc:691] Time spent creating directory manager: real 0.011s user 0.002s sys 0.008s
I20260811 19:01:27.282953 23832 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:01:27.286078 23808 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.002s sys 0.001s
I20260811 19:01:27.286803 23808 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/master-0/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/master-0/wal
uuid: "394f0e9f175e4f24b7e1081aaeb6fb3b"
format_stamp: "Formatted at 2026-08-11 19:01:27 on dist-test-slave-h6n0"
server_key: "54b1f8faf9111f2625fb7e8489539574"
server_key_iv: "69b659430a273751b5130371a7442b26"
server_key_version: "encryptionkey@0"
I20260811 19:01:27.287307 23808 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:01:27.320678 23808 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:01:27.331779 23808 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:01:27.332223 23808 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:01:27.371136 23808 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.190:45339
I20260811 19:01:27.371165 23907 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.190:45339 every 8 connection(s)
I20260811 19:01:27.372656 23808 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
I20260811 19:01:27.374725 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 23808
I20260811 19:01:27.374992 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/master-0/wal/instance
I20260811 19:01:27.376035 30146 external_mini_cluster.cc:1428] Setting key 7e9bd2d0d33b350c0fd154aea379bf5e
I20260811 19:01:27.378021 23908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 19:01:27 dist-test-slave-h6n0 krb5kdc[23787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474886, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 19:01:27.394583 23908 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 394f0e9f175e4f24b7e1081aaeb6fb3b: Bootstrap starting.
I20260811 19:01:27.399258 23908 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 394f0e9f175e4f24b7e1081aaeb6fb3b: Neither blocks nor log segments found. Creating new log.
I20260811 19:01:27.400745 23908 log.cc:824] T 00000000000000000000000000000000 P 394f0e9f175e4f24b7e1081aaeb6fb3b: Log is configured to *not* fsync() on all Append() calls
I20260811 19:01:27.404112 23908 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 394f0e9f175e4f24b7e1081aaeb6fb3b: No bootstrap required, opened a new log
I20260811 19:01:27.405476 23911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:27.378496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42238 (local address 127.29.112.190:45339)
0811 19:01:27.379692 (+ 1196us) server_negotiation.cc:206] Beginning negotiation
0811 19:01:27.379711 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 19:01:27.379776 (+ 65us) server_negotiation.cc:407] Connection header received
0811 19:01:27.381824 (+ 2048us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:01:27.381886 (+ 62us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:01:27.382456 (+ 570us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:01:27.383640 (+ 1184us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:01:27.385787 (+ 2147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:27.387470 (+ 1683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:27.389236 (+ 1766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:27.390262 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:27.390604 (+ 342us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:27.395788 (+ 5184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:01:27.395800 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:01:27.395819 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:01:27.395875 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:01:27.400908 (+ 5033us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:01:27.401744 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:01:27.401753 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:01:27.401761 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:01:27.402009 (+ 248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:01:27.402470 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:01:27.402474 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:01:27.402477 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:01:27.403128 (+ 651us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:01:27.403471 (+ 343us) server_negotiation.cc:1035] Waiting for connection context
0811 19:01:27.403932 (+ 461us) server_negotiation.cc:299] Negotiation successful
0811 19:01:27.404412 (+ 480us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":820,"thread_start_us":502,"threads_started":1}
I20260811 19:01:27.415396 23908 raft_consensus.cc:348] T 00000000000000000000000000000000 P 394f0e9f175e4f24b7e1081aaeb6fb3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "394f0e9f175e4f24b7e1081aaeb6fb3b" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 45339 } }
I20260811 19:01:27.415902 23908 raft_consensus.cc:374] T 00000000000000000000000000000000 P 394f0e9f175e4f24b7e1081aaeb6fb3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:01:27.416065 23908 raft_consensus.cc:729] T 00000000000000000000000000000000 P 394f0e9f175e4f24b7e1081aaeb6fb3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 394f0e9f175e4f24b7e1081aaeb6fb3b, State: Initialized, Role: FOLLOWER
I20260811 19:01:27.417057 23908 consensus_queue.cc:260] T 00000000000000000000000000000000 P 394f0e9f175e4f24b7e1081aaeb6fb3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "394f0e9f175e4f24b7e1081aaeb6fb3b" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 45339 } }
I20260811 19:01:27.417585 23908 raft_consensus.cc:388] T 00000000000000000000000000000000 P 394f0e9f175e4f24b7e1081aaeb6fb3b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:01:27.417776 23908 raft_consensus.cc:482] T 00000000000000000000000000000000 P 394f0e9f175e4f24b7e1081aaeb6fb3b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:01:27.417979 23908 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 394f0e9f175e4f24b7e1081aaeb6fb3b [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:01:27.420683 23908 raft_consensus.cc:504] T 00000000000000000000000000000000 P 394f0e9f175e4f24b7e1081aaeb6fb3b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "394f0e9f175e4f24b7e1081aaeb6fb3b" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 45339 } }
I20260811 19:01:27.421308 23908 leader_election.cc:302] T 00000000000000000000000000000000 P 394f0e9f175e4f24b7e1081aaeb6fb3b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 394f0e9f175e4f24b7e1081aaeb6fb3b; no voters:
I20260811 19:01:27.422000 23908 leader_election.cc:288] T 00000000000000000000000000000000 P 394f0e9f175e4f24b7e1081aaeb6fb3b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:01:27.422196 23914 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 394f0e9f175e4f24b7e1081aaeb6fb3b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:01:27.423732 23914 raft_consensus.cc:686] T 00000000000000000000000000000000 P 394f0e9f175e4f24b7e1081aaeb6fb3b [term 1 LEADER]: Becoming Leader. State: Replica: 394f0e9f175e4f24b7e1081aaeb6fb3b, State: Running, Role: LEADER
I20260811 19:01:27.424443 23908 sys_catalog.cc:564] T 00000000000000000000000000000000 P 394f0e9f175e4f24b7e1081aaeb6fb3b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:01:27.424388 23914 consensus_queue.cc:237] T 00000000000000000000000000000000 P 394f0e9f175e4f24b7e1081aaeb6fb3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "394f0e9f175e4f24b7e1081aaeb6fb3b" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 45339 } }
I20260811 19:01:27.427897 23916 sys_catalog.cc:455] T 00000000000000000000000000000000 P 394f0e9f175e4f24b7e1081aaeb6fb3b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 394f0e9f175e4f24b7e1081aaeb6fb3b. Latest consensus state: current_term: 1 leader_uuid: "394f0e9f175e4f24b7e1081aaeb6fb3b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "394f0e9f175e4f24b7e1081aaeb6fb3b" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 45339 } } }
I20260811 19:01:27.428517 23916 sys_catalog.cc:458] T 00000000000000000000000000000000 P 394f0e9f175e4f24b7e1081aaeb6fb3b [sys.catalog]: This master's current role is: LEADER
I20260811 19:01:27.428463 23915 sys_catalog.cc:455] T 00000000000000000000000000000000 P 394f0e9f175e4f24b7e1081aaeb6fb3b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "394f0e9f175e4f24b7e1081aaeb6fb3b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "394f0e9f175e4f24b7e1081aaeb6fb3b" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 45339 } } }
I20260811 19:01:27.428810 23915 sys_catalog.cc:458] T 00000000000000000000000000000000 P 394f0e9f175e4f24b7e1081aaeb6fb3b [sys.catalog]: This master's current role is: LEADER
I20260811 19:01:27.430780 23920 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:01:27.437048 23920 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:01:27.450850 23920 catalog_manager.cc:1347] Generated new cluster ID: 79b877f1f5ac49f4907b1d7cb27744fa
I20260811 19:01:27.451197 23920 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:01:27.467782 23920 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:01:27.469787 23920 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:01:27.482028 23920 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 394f0e9f175e4f24b7e1081aaeb6fb3b: Generated new TSK 0
I20260811 19:01:27.483605 23920 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.112.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.129.keytab.
Entry for principal kudu/127.29.112.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.129.keytab.
I20260811 19:01:27.581277 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.129:0
--local_ip_for_outbound_sockets=127.29.112.129
--webserver_interface=127.29.112.129
--webserver_port=0
--tserver_master_addrs=127.29.112.190:45339
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.129.keytab
--principal=kudu/127.29.112.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:36075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:01:28.031843 23943 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:01:28.032198 23943 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:01:28.032377 23943 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:01:28.041882 23943 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:01:28.042140 23943 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:01:28.042460 23943 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.129
I20260811 19:01:28.056943 23943 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:36075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.129.keytab
--principal=kudu/127.29.112.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.112.129
--webserver_port=0
--tserver_master_addrs=127.29.112.190:45339
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.129
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:01:28.058619 23943 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:01:28.060272 23943 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:01:28.076304 23951 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:01:28.076643 23955 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:01:28.076498 23953 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:01:28.077795 23943 server_base.cc:1034] running on GCE node
I20260811 19:01:28.078635 23943 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:01:28.079808 23943 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:01:28.081051 23943 hybrid_clock.cc:648] HybridClock initialized: now 1786474888081006 us; error 30 us; skew 500 ppm
I20260811 19:01:28.081418 23943 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:01:28 dist-test-slave-h6n0 krb5kdc[23787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474888, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:01:28.092836 23943 init.cc:377] Logged in from keytab as kudu/127.29.112.129@KRBTEST.COM (short username kudu)
I20260811 19:01:28.096028 23943 webserver.cc:460] Webserver started at http://127.29.112.129:45245/ using document root <none> and password file <none>
I20260811 19:01:28.097186 23943 fs_manager.cc:359] Metadata directory not provided
I20260811 19:01:28.097379 23943 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:01:28.097761 23943 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:01:28.101089 23943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-0/data/instance:
uuid: "87495242ca044b7abcd73e35b8200e67"
format_stamp: "Formatted at 2026-08-11 19:01:28 on dist-test-slave-h6n0"
server_key: "e3e7ca1842fbc341318292979be8d2b0"
server_key_iv: "778c5330775e38609965abf788fc5b59"
server_key_version: "encryptionkey@0"
I20260811 19:01:28.101894 23943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-0/wal/instance:
uuid: "87495242ca044b7abcd73e35b8200e67"
format_stamp: "Formatted at 2026-08-11 19:01:28 on dist-test-slave-h6n0"
server_key: "e3e7ca1842fbc341318292979be8d2b0"
server_key_iv: "778c5330775e38609965abf788fc5b59"
server_key_version: "encryptionkey@0"
I20260811 19:01:28.107697 23943 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260811 19:01:28.114068 23964 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:01:28.116789 23943 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.001s
I20260811 19:01:28.117045 23943 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-0/wal
uuid: "87495242ca044b7abcd73e35b8200e67"
format_stamp: "Formatted at 2026-08-11 19:01:28 on dist-test-slave-h6n0"
server_key: "e3e7ca1842fbc341318292979be8d2b0"
server_key_iv: "778c5330775e38609965abf788fc5b59"
server_key_version: "encryptionkey@0"
I20260811 19:01:28.117303 23943 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:01:28.144575 23943 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:01:28.150938 23943 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:01:28.151273 23943 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:01:28.152783 23943 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:01:28.155076 23943 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:01:28.155186 23943 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:01:28.155580 23943 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:01:28.155689 23943 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:01:28.208138 23943 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.129:46535
I20260811 19:01:28.208182 24135 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.129:46535 every 8 connection(s)
I20260811 19:01:28.210047 23943 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-0/data/info.pb
I20260811 19:01:28.219725 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 23943
I20260811 19:01:28.220029 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-0/wal/instance
I20260811 19:01:28.221068 30146 external_mini_cluster.cc:1428] Setting key c9cde03268d1e96b1ba8b8bdb1c2f89a
Aug 11 19:01:28 dist-test-slave-h6n0 krb5kdc[23787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474888, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.129@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 19:01:28.252187 24138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:28.213862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.129:33263 (local address 127.29.112.190:45339)
0811 19:01:28.214679 (+ 817us) server_negotiation.cc:206] Beginning negotiation
0811 19:01:28.214685 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:01:28.216575 (+ 1890us) server_negotiation.cc:407] Connection header received
0811 19:01:28.220873 (+ 4298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:01:28.220881 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:01:28.221028 (+ 147us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:01:28.221747 (+ 719us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:01:28.225974 (+ 4227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:28.227231 (+ 1257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:28.229309 (+ 2078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:28.229814 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:28.229976 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:28.237139 (+ 7163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:01:28.237146 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:01:28.237150 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:01:28.237194 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:01:28.242217 (+ 5023us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:01:28.244632 (+ 2415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:01:28.244637 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:01:28.244640 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:01:28.244828 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:01:28.246114 (+ 1286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:01:28.246118 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:01:28.246121 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:01:28.248185 (+ 2064us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:01:28.248478 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0811 19:01:28.251170 (+ 2692us) server_negotiation.cc:299] Negotiation successful
0811 19:01:28.251637 (+ 467us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":610,"thread_start_us":461,"threads_started":1}
I20260811 19:01:28.255409 24140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:28.214981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:45339 (local address 127.29.112.129:33263)
0811 19:01:28.216212 (+ 1231us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:28.216296 (+ 84us) client_negotiation.cc:174] Beginning negotiation
0811 19:01:28.220302 (+ 4006us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:01:28.222470 (+ 2168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:01:28.222484 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:01:28.223598 (+ 1114us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:01:28.225639 (+ 2041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:01:28.225681 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:01:28.227519 (+ 1838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:01:28.227525 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:01:28.229102 (+ 1577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:01:28.229122 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:01:28.230120 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:01:28.230130 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:01:28.230411 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:28.231300 (+ 889us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:01:28.231328 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:01:28.236545 (+ 5217us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:01:28.242482 (+ 5937us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:01:28.242514 (+ 32us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:01:28.242577 (+ 63us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:01:28.244291 (+ 1714us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:01:28.245146 (+ 855us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:01:28.245309 (+ 163us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:01:28.245319 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:01:28.245843 (+ 524us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:01:28.248543 (+ 2700us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:01:28.248560 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:01:28.249601 (+ 1041us) client_negotiation.cc:769] Sending connection context
0811 19:01:28.250742 (+ 1141us) client_negotiation.cc:240] Negotiation successful
0811 19:01:28.252369 (+ 1627us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":835,"thread_start_us":429,"threads_started":1}
I20260811 19:01:28.258373 24136 heartbeater.cc:347] Connected to a master server at 127.29.112.190:45339
I20260811 19:01:28.259137 24136 heartbeater.cc:464] Registering TS with master...
I20260811 19:01:28.261264 24136 heartbeater.cc:511] Master 127.29.112.190:45339 requested a full tablet report, sending...
I20260811 19:01:28.265699 23859 ts_manager.cc:194] Registered new tserver with Master: 87495242ca044b7abcd73e35b8200e67 (127.29.112.129:46535)
I20260811 19:01:28.268708 23859 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.129@KRBTEST.COM'} at 127.29.112.129:33263
WARNING: no policy specified for kudu/127.29.112.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.130.keytab.
Entry for principal kudu/127.29.112.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.130.keytab.
W20260811 19:01:28.335604 30146 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.29.112.130: real 0.110s user 0.000s sys 0.035s
I20260811 19:01:28.337119 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.130:0
--local_ip_for_outbound_sockets=127.29.112.130
--webserver_interface=127.29.112.130
--webserver_port=0
--tserver_master_addrs=127.29.112.190:45339
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.130.keytab
--principal=kudu/127.29.112.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:36075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:01:28.817629 24146 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:01:28.818164 24146 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:01:28.818351 24146 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:01:28.829129 24146 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:01:28.829696 24146 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:01:28.829959 24146 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.130
I20260811 19:01:28.848536 24146 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:36075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.130.keytab
--principal=kudu/127.29.112.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.112.130
--webserver_port=0
--tserver_master_addrs=127.29.112.190:45339
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.130
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:01:28.851046 24146 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:01:28.853286 24146 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:01:28.873304 24153 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:01:28.873402 24154 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:01:28.873353 24156 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:01:28.874873 24146 server_base.cc:1034] running on GCE node
I20260811 19:01:28.876055 24146 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:01:28.877553 24146 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:01:28.878913 24146 hybrid_clock.cc:648] HybridClock initialized: now 1786474888878826 us; error 105 us; skew 500 ppm
I20260811 19:01:28.879444 24146 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:01:28 dist-test-slave-h6n0 krb5kdc[23787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474888, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:01:28.883275 24146 init.cc:377] Logged in from keytab as kudu/127.29.112.130@KRBTEST.COM (short username kudu)
I20260811 19:01:28.885522 24146 webserver.cc:460] Webserver started at http://127.29.112.130:33725/ using document root <none> and password file <none>
I20260811 19:01:28.886310 24146 fs_manager.cc:359] Metadata directory not provided
I20260811 19:01:28.886683 24146 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:01:28.889030 24146 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:01:28.893379 24146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-1/data/instance:
uuid: "7b99824f1c6744f29b3c2787495f631c"
format_stamp: "Formatted at 2026-08-11 19:01:28 on dist-test-slave-h6n0"
server_key: "5d7517d24d4f31e365c04e607ea07445"
server_key_iv: "205d31e6af17ae27cd13e7d9574e79cc"
server_key_version: "encryptionkey@0"
I20260811 19:01:28.894245 24146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-1/wal/instance:
uuid: "7b99824f1c6744f29b3c2787495f631c"
format_stamp: "Formatted at 2026-08-11 19:01:28 on dist-test-slave-h6n0"
server_key: "5d7517d24d4f31e365c04e607ea07445"
server_key_iv: "205d31e6af17ae27cd13e7d9574e79cc"
server_key_version: "encryptionkey@0"
I20260811 19:01:28.903537 24146 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.007s sys 0.000s
I20260811 19:01:28.909194 24168 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:01:28.912863 24146 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.002s sys 0.000s
I20260811 19:01:28.913261 24146 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-1/wal
uuid: "7b99824f1c6744f29b3c2787495f631c"
format_stamp: "Formatted at 2026-08-11 19:01:28 on dist-test-slave-h6n0"
server_key: "5d7517d24d4f31e365c04e607ea07445"
server_key_iv: "205d31e6af17ae27cd13e7d9574e79cc"
server_key_version: "encryptionkey@0"
I20260811 19:01:28.913568 24146 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:01:28.950771 24146 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:01:28.959640 24146 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:01:28.960052 24146 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:01:28.961293 24146 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:01:28.963069 24146 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:01:28.963164 24146 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:01:28.963300 24146 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:01:28.963387 24146 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:01:29.016602 24146 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.130:37991
I20260811 19:01:29.016642 24348 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.130:37991 every 8 connection(s)
I20260811 19:01:29.018432 24146 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-1/data/info.pb
I20260811 19:01:29.029505 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 24146
I20260811 19:01:29.029874 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-1/wal/instance
I20260811 19:01:29.030920 30146 external_mini_cluster.cc:1428] Setting key 775f3df867651bc94fea644a548a5e6f
Aug 11 19:01:29 dist-test-slave-h6n0 krb5kdc[23787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474888, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.130@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 19:01:29.050341 24353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:29.022407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.130:55921 (local address 127.29.112.190:45339)
0811 19:01:29.023485 (+ 1078us) server_negotiation.cc:206] Beginning negotiation
0811 19:01:29.023490 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:01:29.024875 (+ 1385us) server_negotiation.cc:407] Connection header received
0811 19:01:29.027738 (+ 2863us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:01:29.027745 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:01:29.027877 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:01:29.028026 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:01:29.030315 (+ 2289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:29.031404 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:29.033746 (+ 2342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:29.034911 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:29.035058 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:29.040427 (+ 5369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:01:29.040434 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:01:29.040438 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:01:29.040484 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:01:29.044692 (+ 4208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:01:29.045701 (+ 1009us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:01:29.045715 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:01:29.045718 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:01:29.045861 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:01:29.046937 (+ 1076us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:01:29.046944 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:01:29.046947 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:01:29.047630 (+ 683us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:01:29.048043 (+ 413us) server_negotiation.cc:1035] Waiting for connection context
0811 19:01:29.049491 (+ 1448us) server_negotiation.cc:299] Negotiation successful
0811 19:01:29.049882 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":750,"thread_start_us":473,"threads_started":1}
I20260811 19:01:29.051815 24354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:29.023207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:45339 (local address 127.29.112.130:55921)
0811 19:01:29.024677 (+ 1470us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:29.024722 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 19:01:29.027251 (+ 2529us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:01:29.028417 (+ 1166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:01:29.028428 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:01:29.029167 (+ 739us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:01:29.030087 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:01:29.030116 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:01:29.031675 (+ 1559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:01:29.031681 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:01:29.033520 (+ 1839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:01:29.033542 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:01:29.035221 (+ 1679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:01:29.035229 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:01:29.035619 (+ 390us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:29.036773 (+ 1154us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:01:29.036814 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:01:29.040099 (+ 3285us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:01:29.044927 (+ 4828us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:01:29.044939 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:01:29.044963 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:01:29.045536 (+ 573us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:01:29.046010 (+ 474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:01:29.046022 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:01:29.046025 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:01:29.046641 (+ 616us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:01:29.048008 (+ 1367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:01:29.048020 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:01:29.048809 (+ 789us) client_negotiation.cc:769] Sending connection context
0811 19:01:29.049199 (+ 390us) client_negotiation.cc:240] Negotiation successful
0811 19:01:29.050175 (+ 976us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":963,"thread_start_us":465,"threads_started":1}
I20260811 19:01:29.053897 24349 heartbeater.cc:347] Connected to a master server at 127.29.112.190:45339
I20260811 19:01:29.054590 24349 heartbeater.cc:464] Registering TS with master...
I20260811 19:01:29.056135 24349 heartbeater.cc:511] Master 127.29.112.190:45339 requested a full tablet report, sending...
I20260811 19:01:29.058866 23859 ts_manager.cc:194] Registered new tserver with Master: 7b99824f1c6744f29b3c2787495f631c (127.29.112.130:37991)
I20260811 19:01:29.060408 23859 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.130@KRBTEST.COM'} at 127.29.112.130:55921
WARNING: no policy specified for kudu/127.29.112.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.131.keytab.
Entry for principal kudu/127.29.112.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.131.keytab.
I20260811 19:01:29.124815 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.131:0
--local_ip_for_outbound_sockets=127.29.112.131
--webserver_interface=127.29.112.131
--webserver_port=0
--tserver_master_addrs=127.29.112.190:45339
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.131.keytab
--principal=kudu/127.29.112.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:36075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 19:01:29.274005 24136 heartbeater.cc:503] Master 127.29.112.190:45339 was elected leader, sending a full tablet report...
W20260811 19:01:29.544091 24358 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:01:29.544521 24358 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:01:29.544639 24358 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:01:29.554054 24358 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:01:29.554201 24358 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:01:29.554347 24358 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.131
I20260811 19:01:29.567648 24358 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:36075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.131.keytab
--principal=kudu/127.29.112.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.112.131
--webserver_port=0
--tserver_master_addrs=127.29.112.190:45339
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.131
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:01:29.569123 24358 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:01:29.570585 24358 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:01:29.589517 24371 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:01:29.589488 24365 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:01:29.589461 24366 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:01:29.591498 24358 server_base.cc:1034] running on GCE node
I20260811 19:01:29.592294 24358 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:01:29.593511 24358 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:01:29.594722 24358 hybrid_clock.cc:648] HybridClock initialized: now 1786474889594657 us; error 51 us; skew 500 ppm
I20260811 19:01:29.595119 24358 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:01:29 dist-test-slave-h6n0 krb5kdc[23787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474889, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:01:29.600348 24358 init.cc:377] Logged in from keytab as kudu/127.29.112.131@KRBTEST.COM (short username kudu)
I20260811 19:01:29.602211 24358 webserver.cc:460] Webserver started at http://127.29.112.131:41429/ using document root <none> and password file <none>
I20260811 19:01:29.603091 24358 fs_manager.cc:359] Metadata directory not provided
I20260811 19:01:29.603217 24358 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:01:29.603565 24358 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:01:29.606997 24358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-2/data/instance:
uuid: "d1c2c5c12d01441d843f06f985ca1684"
format_stamp: "Formatted at 2026-08-11 19:01:29 on dist-test-slave-h6n0"
server_key: "391405d26b86085610c85f500acb08b8"
server_key_iv: "35bf95b36cd7cb7ba01aefd9acfcf118"
server_key_version: "encryptionkey@0"
I20260811 19:01:29.607803 24358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-2/wal/instance:
uuid: "d1c2c5c12d01441d843f06f985ca1684"
format_stamp: "Formatted at 2026-08-11 19:01:29 on dist-test-slave-h6n0"
server_key: "391405d26b86085610c85f500acb08b8"
server_key_iv: "35bf95b36cd7cb7ba01aefd9acfcf118"
server_key_version: "encryptionkey@0"
I20260811 19:01:29.613556 24358 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 19:01:29.618119 24378 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:01:29.620442 24358 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260811 19:01:29.620666 24358 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-2/wal
uuid: "d1c2c5c12d01441d843f06f985ca1684"
format_stamp: "Formatted at 2026-08-11 19:01:29 on dist-test-slave-h6n0"
server_key: "391405d26b86085610c85f500acb08b8"
server_key_iv: "35bf95b36cd7cb7ba01aefd9acfcf118"
server_key_version: "encryptionkey@0"
I20260811 19:01:29.620903 24358 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:01:29.646090 24358 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:01:29.652415 24358 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:01:29.652850 24358 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:01:29.654446 24358 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:01:29.656183 24358 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:01:29.656319 24358 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:01:29.656693 24358 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:01:29.656833 24358 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:01:29.705086 24358 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.131:39661
I20260811 19:01:29.705190 24569 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.131:39661 every 8 connection(s)
I20260811 19:01:29.706511 24358 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-2/data/info.pb
I20260811 19:01:29.716600 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 24358
I20260811 19:01:29.716845 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-2/wal/instance
I20260811 19:01:29.717437 30146 external_mini_cluster.cc:1428] Setting key 133e2ff841ac227c3ae2757a20e12292
Aug 11 19:01:29 dist-test-slave-h6n0 krb5kdc[23787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474889, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.131@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 19:01:29.736497 24572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:29.709701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.131:47783 (local address 127.29.112.190:45339)
0811 19:01:29.710274 (+ 573us) server_negotiation.cc:206] Beginning negotiation
0811 19:01:29.710279 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:01:29.712388 (+ 2109us) server_negotiation.cc:407] Connection header received
0811 19:01:29.714593 (+ 2205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:01:29.714600 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:01:29.714727 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:01:29.714881 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:01:29.716591 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:29.717933 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:29.720588 (+ 2655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:29.721365 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:29.721559 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:29.727502 (+ 5943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:01:29.727508 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:01:29.727511 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:01:29.727561 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:01:29.732512 (+ 4951us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:01:29.733616 (+ 1104us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:01:29.733622 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:01:29.733634 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:01:29.733774 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:01:29.734275 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:01:29.734279 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:01:29.734282 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:01:29.734760 (+ 478us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:01:29.735302 (+ 542us) server_negotiation.cc:1035] Waiting for connection context
0811 19:01:29.735762 (+ 460us) server_negotiation.cc:299] Negotiation successful
0811 19:01:29.735966 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":457,"thread_start_us":321,"threads_started":1}
I20260811 19:01:29.737929 24573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:29.710802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:45339 (local address 127.29.112.131:47783)
0811 19:01:29.712166 (+ 1364us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:29.712222 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 19:01:29.714246 (+ 2024us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:01:29.715150 (+ 904us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:01:29.715159 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:01:29.715769 (+ 610us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:01:29.716397 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:01:29.716417 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:01:29.718122 (+ 1705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:01:29.718135 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:01:29.720033 (+ 1898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:01:29.720304 (+ 271us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:01:29.721745 (+ 1441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:01:29.721757 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:01:29.722240 (+ 483us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:29.723381 (+ 1141us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:01:29.723412 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:01:29.727139 (+ 3727us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:01:29.732759 (+ 5620us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:01:29.732770 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:01:29.732787 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:01:29.733453 (+ 666us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:01:29.733903 (+ 450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:01:29.733909 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:01:29.733913 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:01:29.734136 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:01:29.734918 (+ 782us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:01:29.734925 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:01:29.735332 (+ 407us) client_negotiation.cc:769] Sending connection context
0811 19:01:29.735602 (+ 270us) client_negotiation.cc:240] Negotiation successful
0811 19:01:29.736058 (+ 456us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1051,"thread_start_us":592,"threads_started":1}
I20260811 19:01:29.739682 24570 heartbeater.cc:347] Connected to a master server at 127.29.112.190:45339
I20260811 19:01:29.740068 24570 heartbeater.cc:464] Registering TS with master...
I20260811 19:01:29.740885 24570 heartbeater.cc:511] Master 127.29.112.190:45339 requested a full tablet report, sending...
I20260811 19:01:29.743139 23859 ts_manager.cc:194] Registered new tserver with Master: d1c2c5c12d01441d843f06f985ca1684 (127.29.112.131:39661)
I20260811 19:01:29.744045 23859 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.131@KRBTEST.COM'} at 127.29.112.131:47783
I20260811 19:01:29.749037 30146 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 19:01:29.771600 24572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:29.755024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36808 (local address 127.29.112.190:45339)
0811 19:01:29.755338 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0811 19:01:29.755343 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:01:29.755366 (+ 23us) server_negotiation.cc:407] Connection header received
0811 19:01:29.755560 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:01:29.755565 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:01:29.755649 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:01:29.755749 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:01:29.758071 (+ 2322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:29.759016 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:29.760709 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:29.761259 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:29.761407 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:29.764720 (+ 3313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:01:29.764726 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:01:29.764730 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:01:29.764781 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:01:29.768443 (+ 3662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:01:29.769327 (+ 884us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:01:29.769333 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:01:29.769336 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:01:29.769543 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:01:29.770086 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:01:29.770091 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:01:29.770094 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:01:29.770508 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:01:29.770896 (+ 388us) server_negotiation.cc:1035] Waiting for connection context
0811 19:01:29.771131 (+ 235us) server_negotiation.cc:299] Negotiation successful
0811 19:01:29.771334 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181}
I20260811 19:01:29.777464 23859 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36808:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 19:01:29.785336 23859 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 19:01:29.831981 24588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:29.803175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49952 (local address 127.29.112.130:37991)
0811 19:01:29.819928 (+ 16753us) server_negotiation.cc:206] Beginning negotiation
0811 19:01:29.819937 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 19:01:29.819973 (+ 36us) server_negotiation.cc:407] Connection header received
0811 19:01:29.820075 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:01:29.820081 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:01:29.820311 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:01:29.820601 (+ 290us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:01:29.821397 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:29.823226 (+ 1829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:29.827612 (+ 4386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:29.829628 (+ 2016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:29.829848 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:29.830055 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0811 19:01:29.830918 (+ 863us) server_negotiation.cc:299] Negotiation successful
0811 19:01:29.831516 (+ 598us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16554,"thread_start_us":16382,"threads_started":1}
I20260811 19:01:29.832075 24585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:29.802116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.130:37991 (local address 127.0.0.1:49952)
0811 19:01:29.817307 (+ 15191us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:29.817356 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 19:01:29.817540 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:01:29.820605 (+ 3065us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:01:29.820614 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:01:29.820661 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:01:29.821153 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:01:29.821168 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:01:29.823399 (+ 2231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:01:29.823404 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:01:29.827287 (+ 3883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:01:29.827305 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:01:29.829965 (+ 2660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:01:29.829974 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:01:29.830401 (+ 427us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:29.830512 (+ 111us) client_negotiation.cc:769] Sending connection context
0811 19:01:29.830727 (+ 215us) client_negotiation.cc:240] Negotiation successful
0811 19:01:29.831448 (+ 721us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14989,"mutex_wait_us":2,"thread_start_us":1407,"threads_started":1}
I20260811 19:01:29.832813 24587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:29.801434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:46535 (local address 127.0.0.1:56580)
0811 19:01:29.803086 (+ 1652us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:29.803146 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 19:01:29.803371 (+ 225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:01:29.822302 (+ 18931us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:01:29.822310 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:01:29.822348 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:01:29.822901 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:01:29.822939 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:01:29.825502 (+ 2563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:01:29.825508 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:01:29.828652 (+ 3144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:01:29.828687 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:01:29.831685 (+ 2998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:01:29.831694 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:01:29.831927 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:29.831973 (+ 46us) client_negotiation.cc:769] Sending connection context
0811 19:01:29.832137 (+ 164us) client_negotiation.cc:240] Negotiation successful
0811 19:01:29.832301 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1300,"mutex_wait_us":23,"thread_start_us":15475,"threads_started":1}
I20260811 19:01:29.833092 24591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:29.805143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46814 (local address 127.29.112.131:39661)
0811 19:01:29.817752 (+ 12609us) server_negotiation.cc:206] Beginning negotiation
0811 19:01:29.817759 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:01:29.820209 (+ 2450us) server_negotiation.cc:407] Connection header received
0811 19:01:29.820469 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:01:29.820485 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:01:29.822126 (+ 1641us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:01:29.822425 (+ 299us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:01:29.823494 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:29.824900 (+ 1406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:29.827964 (+ 3064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:29.829628 (+ 1664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:29.829860 (+ 232us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:29.830061 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0811 19:01:29.832491 (+ 2430us) server_negotiation.cc:299] Negotiation successful
0811 19:01:29.832688 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12425,"thread_start_us":12229,"threads_started":1}
I20260811 19:01:29.833751 24586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:29.801933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56580 (local address 127.29.112.129:46535)
0811 19:01:29.821613 (+ 19680us) server_negotiation.cc:206] Beginning negotiation
0811 19:01:29.821623 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 19:01:29.821656 (+ 33us) server_negotiation.cc:407] Connection header received
0811 19:01:29.821828 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:01:29.821837 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:01:29.822067 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:01:29.822273 (+ 206us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:01:29.823113 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:29.825304 (+ 2191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:29.829116 (+ 3812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:29.830891 (+ 1775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:29.832801 (+ 1910us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:29.833063 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0811 19:01:29.833173 (+ 110us) server_negotiation.cc:299] Negotiation successful
0811 19:01:29.833382 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19456,"thread_start_us":19117,"threads_started":1}
I20260811 19:01:29.833771 24589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:29.804703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.131:39661 (local address 127.0.0.1:46814)
0811 19:01:29.819140 (+ 14437us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:29.819168 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0811 19:01:29.819306 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:01:29.822699 (+ 3393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:01:29.822705 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:01:29.822733 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:01:29.823276 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:01:29.823293 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:01:29.825048 (+ 1755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:01:29.825055 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:01:29.827583 (+ 2528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:01:29.827598 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:01:29.831341 (+ 3743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:01:29.831359 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:01:29.831585 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:29.831629 (+ 44us) client_negotiation.cc:769] Sending connection context
0811 19:01:29.831781 (+ 152us) client_negotiation.cc:240] Negotiation successful
0811 19:01:29.831963 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14286,"thread_start_us":14089,"threads_started":1}
I20260811 19:01:29.837881 24455 tablet_service.cc:1467] Processing CreateTablet for tablet 4f4db87d32b5453fa0f2636eaf926313 (DEFAULT_TABLE table=test-table [id=95ac80e3a5114483988226925e552289]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:01:29.837904 24041 tablet_service.cc:1467] Processing CreateTablet for tablet 4f4db87d32b5453fa0f2636eaf926313 (DEFAULT_TABLE table=test-table [id=95ac80e3a5114483988226925e552289]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:01:29.837904 24242 tablet_service.cc:1467] Processing CreateTablet for tablet 4f4db87d32b5453fa0f2636eaf926313 (DEFAULT_TABLE table=test-table [id=95ac80e3a5114483988226925e552289]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:01:29.839330 24455 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f4db87d32b5453fa0f2636eaf926313. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:01:29.839339 24041 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f4db87d32b5453fa0f2636eaf926313. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:01:29.839357 24242 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f4db87d32b5453fa0f2636eaf926313. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:01:29.853252 24592 tablet_bootstrap.cc:492] T 4f4db87d32b5453fa0f2636eaf926313 P 87495242ca044b7abcd73e35b8200e67: Bootstrap starting.
I20260811 19:01:29.858203 24593 tablet_bootstrap.cc:492] T 4f4db87d32b5453fa0f2636eaf926313 P d1c2c5c12d01441d843f06f985ca1684: Bootstrap starting.
I20260811 19:01:29.858827 24592 tablet_bootstrap.cc:654] T 4f4db87d32b5453fa0f2636eaf926313 P 87495242ca044b7abcd73e35b8200e67: Neither blocks nor log segments found. Creating new log.
I20260811 19:01:29.860677 24592 log.cc:824] T 4f4db87d32b5453fa0f2636eaf926313 P 87495242ca044b7abcd73e35b8200e67: Log is configured to *not* fsync() on all Append() calls
I20260811 19:01:29.861668 24594 tablet_bootstrap.cc:492] T 4f4db87d32b5453fa0f2636eaf926313 P 7b99824f1c6744f29b3c2787495f631c: Bootstrap starting.
I20260811 19:01:29.863498 24593 tablet_bootstrap.cc:654] T 4f4db87d32b5453fa0f2636eaf926313 P d1c2c5c12d01441d843f06f985ca1684: Neither blocks nor log segments found. Creating new log.
I20260811 19:01:29.864879 24593 log.cc:824] T 4f4db87d32b5453fa0f2636eaf926313 P d1c2c5c12d01441d843f06f985ca1684: Log is configured to *not* fsync() on all Append() calls
I20260811 19:01:29.866119 24594 tablet_bootstrap.cc:654] T 4f4db87d32b5453fa0f2636eaf926313 P 7b99824f1c6744f29b3c2787495f631c: Neither blocks nor log segments found. Creating new log.
I20260811 19:01:29.866914 24592 tablet_bootstrap.cc:492] T 4f4db87d32b5453fa0f2636eaf926313 P 87495242ca044b7abcd73e35b8200e67: No bootstrap required, opened a new log
I20260811 19:01:29.867273 24592 ts_tablet_manager.cc:1397] T 4f4db87d32b5453fa0f2636eaf926313 P 87495242ca044b7abcd73e35b8200e67: Time spent bootstrapping tablet: real 0.014s user 0.007s sys 0.005s
I20260811 19:01:29.867750 24594 log.cc:824] T 4f4db87d32b5453fa0f2636eaf926313 P 7b99824f1c6744f29b3c2787495f631c: Log is configured to *not* fsync() on all Append() calls
I20260811 19:01:29.869194 24593 tablet_bootstrap.cc:492] T 4f4db87d32b5453fa0f2636eaf926313 P d1c2c5c12d01441d843f06f985ca1684: No bootstrap required, opened a new log
I20260811 19:01:29.869493 24593 ts_tablet_manager.cc:1397] T 4f4db87d32b5453fa0f2636eaf926313 P d1c2c5c12d01441d843f06f985ca1684: Time spent bootstrapping tablet: real 0.012s user 0.010s sys 0.000s
I20260811 19:01:29.873266 24594 tablet_bootstrap.cc:492] T 4f4db87d32b5453fa0f2636eaf926313 P 7b99824f1c6744f29b3c2787495f631c: No bootstrap required, opened a new log
I20260811 19:01:29.873611 24594 ts_tablet_manager.cc:1397] T 4f4db87d32b5453fa0f2636eaf926313 P 7b99824f1c6744f29b3c2787495f631c: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.005s
I20260811 19:01:29.880692 24592 raft_consensus.cc:348] T 4f4db87d32b5453fa0f2636eaf926313 P 87495242ca044b7abcd73e35b8200e67 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87495242ca044b7abcd73e35b8200e67" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 46535 } } peers { permanent_uuid: "7b99824f1c6744f29b3c2787495f631c" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 37991 } } peers { permanent_uuid: "d1c2c5c12d01441d843f06f985ca1684" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 39661 } }
I20260811 19:01:29.881608 24592 raft_consensus.cc:374] T 4f4db87d32b5453fa0f2636eaf926313 P 87495242ca044b7abcd73e35b8200e67 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:01:29.881824 24592 raft_consensus.cc:729] T 4f4db87d32b5453fa0f2636eaf926313 P 87495242ca044b7abcd73e35b8200e67 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87495242ca044b7abcd73e35b8200e67, State: Initialized, Role: FOLLOWER
I20260811 19:01:29.882006 24593 raft_consensus.cc:348] T 4f4db87d32b5453fa0f2636eaf926313 P d1c2c5c12d01441d843f06f985ca1684 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87495242ca044b7abcd73e35b8200e67" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 46535 } } peers { permanent_uuid: "7b99824f1c6744f29b3c2787495f631c" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 37991 } } peers { permanent_uuid: "d1c2c5c12d01441d843f06f985ca1684" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 39661 } }
I20260811 19:01:29.882513 24593 raft_consensus.cc:374] T 4f4db87d32b5453fa0f2636eaf926313 P d1c2c5c12d01441d843f06f985ca1684 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:01:29.882673 24593 raft_consensus.cc:729] T 4f4db87d32b5453fa0f2636eaf926313 P d1c2c5c12d01441d843f06f985ca1684 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1c2c5c12d01441d843f06f985ca1684, State: Initialized, Role: FOLLOWER
I20260811 19:01:29.882242 24594 raft_consensus.cc:348] T 4f4db87d32b5453fa0f2636eaf926313 P 7b99824f1c6744f29b3c2787495f631c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87495242ca044b7abcd73e35b8200e67" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 46535 } } peers { permanent_uuid: "7b99824f1c6744f29b3c2787495f631c" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 37991 } } peers { permanent_uuid: "d1c2c5c12d01441d843f06f985ca1684" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 39661 } }
I20260811 19:01:29.882858 24592 consensus_queue.cc:260] T 4f4db87d32b5453fa0f2636eaf926313 P 87495242ca044b7abcd73e35b8200e67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87495242ca044b7abcd73e35b8200e67" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 46535 } } peers { permanent_uuid: "7b99824f1c6744f29b3c2787495f631c" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 37991 } } peers { permanent_uuid: "d1c2c5c12d01441d843f06f985ca1684" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 39661 } }
I20260811 19:01:29.883074 24594 raft_consensus.cc:374] T 4f4db87d32b5453fa0f2636eaf926313 P 7b99824f1c6744f29b3c2787495f631c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:01:29.883195 24594 raft_consensus.cc:729] T 4f4db87d32b5453fa0f2636eaf926313 P 7b99824f1c6744f29b3c2787495f631c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b99824f1c6744f29b3c2787495f631c, State: Initialized, Role: FOLLOWER
I20260811 19:01:29.883466 24593 consensus_queue.cc:260] T 4f4db87d32b5453fa0f2636eaf926313 P d1c2c5c12d01441d843f06f985ca1684 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87495242ca044b7abcd73e35b8200e67" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 46535 } } peers { permanent_uuid: "7b99824f1c6744f29b3c2787495f631c" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 37991 } } peers { permanent_uuid: "d1c2c5c12d01441d843f06f985ca1684" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 39661 } }
I20260811 19:01:29.883929 24594 consensus_queue.cc:260] T 4f4db87d32b5453fa0f2636eaf926313 P 7b99824f1c6744f29b3c2787495f631c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87495242ca044b7abcd73e35b8200e67" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 46535 } } peers { permanent_uuid: "7b99824f1c6744f29b3c2787495f631c" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 37991 } } peers { permanent_uuid: "d1c2c5c12d01441d843f06f985ca1684" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 39661 } }
I20260811 19:01:29.887395 24592 ts_tablet_manager.cc:1428] T 4f4db87d32b5453fa0f2636eaf926313 P 87495242ca044b7abcd73e35b8200e67: Time spent starting tablet: real 0.019s user 0.017s sys 0.000s
I20260811 19:01:29.888758 24349 heartbeater.cc:503] Master 127.29.112.190:45339 was elected leader, sending a full tablet report...
I20260811 19:01:29.889418 24594 ts_tablet_manager.cc:1428] T 4f4db87d32b5453fa0f2636eaf926313 P 7b99824f1c6744f29b3c2787495f631c: Time spent starting tablet: real 0.016s user 0.008s sys 0.007s
I20260811 19:01:29.890951 24570 heartbeater.cc:503] Master 127.29.112.190:45339 was elected leader, sending a full tablet report...
I20260811 19:01:29.893154 24593 ts_tablet_manager.cc:1428] T 4f4db87d32b5453fa0f2636eaf926313 P d1c2c5c12d01441d843f06f985ca1684: Time spent starting tablet: real 0.024s user 0.012s sys 0.009s
W20260811 19:01:29.958726 24571 tablet.cc:2369] T 4f4db87d32b5453fa0f2636eaf926313 P d1c2c5c12d01441d843f06f985ca1684: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 19:01:29.963979 24137 tablet.cc:2369] T 4f4db87d32b5453fa0f2636eaf926313 P 87495242ca044b7abcd73e35b8200e67: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 19:01:29.994710 24598 raft_consensus.cc:482] T 4f4db87d32b5453fa0f2636eaf926313 P 87495242ca044b7abcd73e35b8200e67 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:01:29.995211 24598 raft_consensus.cc:504] T 4f4db87d32b5453fa0f2636eaf926313 P 87495242ca044b7abcd73e35b8200e67 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87495242ca044b7abcd73e35b8200e67" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 46535 } } peers { permanent_uuid: "7b99824f1c6744f29b3c2787495f631c" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 37991 } } peers { permanent_uuid: "d1c2c5c12d01441d843f06f985ca1684" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 39661 } }
I20260811 19:01:29.997191 24598 leader_election.cc:288] T 4f4db87d32b5453fa0f2636eaf926313 P 87495242ca044b7abcd73e35b8200e67 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7b99824f1c6744f29b3c2787495f631c (127.29.112.130:37991), d1c2c5c12d01441d843f06f985ca1684 (127.29.112.131:39661)
I20260811 19:01:30.006981 24588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:29.997867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.129:36909 (local address 127.29.112.130:37991)
0811 19:01:29.998183 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0811 19:01:29.998191 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:01:29.999326 (+ 1135us) server_negotiation.cc:407] Connection header received
0811 19:01:29.999754 (+ 428us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:01:29.999759 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:01:29.999825 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:01:29.999904 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:01:30.000846 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:30.001756 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:30.004235 (+ 2479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:30.005552 (+ 1317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:30.005702 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:30.005783 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 19:01:30.006401 (+ 618us) server_negotiation.cc:299] Negotiation successful
0811 19:01:30.006674 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109}
I20260811 19:01:30.007177 24603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:29.997566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.130:37991 (local address 127.29.112.129:36909)
0811 19:01:29.999054 (+ 1488us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:29.999089 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0811 19:01:29.999533 (+ 444us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:01:30.000164 (+ 631us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:01:30.000169 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:01:30.000196 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:01:30.000660 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:01:30.000672 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:01:30.001920 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:01:30.001927 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:01:30.003962 (+ 2035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:01:30.003977 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:01:30.005839 (+ 1862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:01:30.005847 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:01:30.006065 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:30.006111 (+ 46us) client_negotiation.cc:769] Sending connection context
0811 19:01:30.006257 (+ 146us) client_negotiation.cc:240] Negotiation successful
0811 19:01:30.006460 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1212,"mutex_wait_us":1,"thread_start_us":690,"threads_started":1}
I20260811 19:01:30.008352 24591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:29.998859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.129:39767 (local address 127.29.112.131:39661)
0811 19:01:29.999124 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0811 19:01:29.999132 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:01:29.999941 (+ 809us) server_negotiation.cc:407] Connection header received
0811 19:01:30.000112 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:01:30.000118 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:01:30.000198 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:01:30.000293 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:01:30.001307 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:30.002565 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:30.005388 (+ 2823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:30.006812 (+ 1424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:30.007011 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:30.007108 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 19:01:30.007951 (+ 843us) server_negotiation.cc:299] Negotiation successful
0811 19:01:30.008152 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260811 19:01:30.008613 24604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:29.998757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.131:39661 (local address 127.29.112.129:39767)
0811 19:01:29.999793 (+ 1036us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:29.999817 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 19:01:29.999964 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:01:30.000360 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:01:30.000365 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:01:30.000400 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:01:30.000909 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:01:30.000927 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:01:30.002829 (+ 1902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:01:30.002833 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:01:30.005074 (+ 2241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:01:30.005103 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:01:30.007451 (+ 2348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:01:30.007462 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:01:30.007698 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:30.007734 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 19:01:30.007841 (+ 107us) client_negotiation.cc:240] Negotiation successful
0811 19:01:30.008022 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":890,"mutex_wait_us":83,"thread_start_us":508,"threads_started":1}
I20260811 19:01:30.008658 24279 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f4db87d32b5453fa0f2636eaf926313" candidate_uuid: "87495242ca044b7abcd73e35b8200e67" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b99824f1c6744f29b3c2787495f631c" is_pre_election: true
I20260811 19:01:30.009387 24279 raft_consensus.cc:2454] T 4f4db87d32b5453fa0f2636eaf926313 P 7b99824f1c6744f29b3c2787495f631c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87495242ca044b7abcd73e35b8200e67 in term 0.
I20260811 19:01:30.009847 24486 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f4db87d32b5453fa0f2636eaf926313" candidate_uuid: "87495242ca044b7abcd73e35b8200e67" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1c2c5c12d01441d843f06f985ca1684" is_pre_election: true
I20260811 19:01:30.010425 24486 raft_consensus.cc:2454] T 4f4db87d32b5453fa0f2636eaf926313 P d1c2c5c12d01441d843f06f985ca1684 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87495242ca044b7abcd73e35b8200e67 in term 0.
I20260811 19:01:30.010486 23969 leader_election.cc:302] T 4f4db87d32b5453fa0f2636eaf926313 P 87495242ca044b7abcd73e35b8200e67 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7b99824f1c6744f29b3c2787495f631c, 87495242ca044b7abcd73e35b8200e67; no voters:
I20260811 19:01:30.011252 24598 raft_consensus.cc:2781] T 4f4db87d32b5453fa0f2636eaf926313 P 87495242ca044b7abcd73e35b8200e67 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 19:01:30.011596 24598 raft_consensus.cc:482] T 4f4db87d32b5453fa0f2636eaf926313 P 87495242ca044b7abcd73e35b8200e67 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 19:01:30.011754 24598 raft_consensus.cc:3037] T 4f4db87d32b5453fa0f2636eaf926313 P 87495242ca044b7abcd73e35b8200e67 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:01:30.014271 24598 raft_consensus.cc:504] T 4f4db87d32b5453fa0f2636eaf926313 P 87495242ca044b7abcd73e35b8200e67 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87495242ca044b7abcd73e35b8200e67" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 46535 } } peers { permanent_uuid: "7b99824f1c6744f29b3c2787495f631c" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 37991 } } peers { permanent_uuid: "d1c2c5c12d01441d843f06f985ca1684" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 39661 } }
I20260811 19:01:30.015215 24598 leader_election.cc:288] T 4f4db87d32b5453fa0f2636eaf926313 P 87495242ca044b7abcd73e35b8200e67 [CANDIDATE]: Term 1 election: Requested vote from peers 7b99824f1c6744f29b3c2787495f631c (127.29.112.130:37991), d1c2c5c12d01441d843f06f985ca1684 (127.29.112.131:39661)
I20260811 19:01:30.015995 24279 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f4db87d32b5453fa0f2636eaf926313" candidate_uuid: "87495242ca044b7abcd73e35b8200e67" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b99824f1c6744f29b3c2787495f631c"
I20260811 19:01:30.016079 24486 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f4db87d32b5453fa0f2636eaf926313" candidate_uuid: "87495242ca044b7abcd73e35b8200e67" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1c2c5c12d01441d843f06f985ca1684"
I20260811 19:01:30.016568 24279 raft_consensus.cc:3037] T 4f4db87d32b5453fa0f2636eaf926313 P 7b99824f1c6744f29b3c2787495f631c [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:01:30.016633 24486 raft_consensus.cc:3037] T 4f4db87d32b5453fa0f2636eaf926313 P d1c2c5c12d01441d843f06f985ca1684 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:01:30.020838 24486 raft_consensus.cc:2454] T 4f4db87d32b5453fa0f2636eaf926313 P d1c2c5c12d01441d843f06f985ca1684 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87495242ca044b7abcd73e35b8200e67 in term 1.
I20260811 19:01:30.021199 24279 raft_consensus.cc:2454] T 4f4db87d32b5453fa0f2636eaf926313 P 7b99824f1c6744f29b3c2787495f631c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87495242ca044b7abcd73e35b8200e67 in term 1.
W20260811 19:01:30.021317 24352 tablet.cc:2369] T 4f4db87d32b5453fa0f2636eaf926313 P 7b99824f1c6744f29b3c2787495f631c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 19:01:30.021625 23969 leader_election.cc:302] T 4f4db87d32b5453fa0f2636eaf926313 P 87495242ca044b7abcd73e35b8200e67 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 87495242ca044b7abcd73e35b8200e67, d1c2c5c12d01441d843f06f985ca1684; no voters:
I20260811 19:01:30.022336 24598 raft_consensus.cc:2781] T 4f4db87d32b5453fa0f2636eaf926313 P 87495242ca044b7abcd73e35b8200e67 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:01:30.023041 24598 raft_consensus.cc:686] T 4f4db87d32b5453fa0f2636eaf926313 P 87495242ca044b7abcd73e35b8200e67 [term 1 LEADER]: Becoming Leader. State: Replica: 87495242ca044b7abcd73e35b8200e67, State: Running, Role: LEADER
I20260811 19:01:30.024115 24598 consensus_queue.cc:237] T 4f4db87d32b5453fa0f2636eaf926313 P 87495242ca044b7abcd73e35b8200e67 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87495242ca044b7abcd73e35b8200e67" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 46535 } } peers { permanent_uuid: "7b99824f1c6744f29b3c2787495f631c" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 37991 } } peers { permanent_uuid: "d1c2c5c12d01441d843f06f985ca1684" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 39661 } }
I20260811 19:01:30.030695 23858 catalog_manager.cc:5495] T 4f4db87d32b5453fa0f2636eaf926313 P 87495242ca044b7abcd73e35b8200e67 reported cstate change: term changed from 0 to 1, leader changed from <none> to 87495242ca044b7abcd73e35b8200e67 (127.29.112.129). New cstate: current_term: 1 leader_uuid: "87495242ca044b7abcd73e35b8200e67" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87495242ca044b7abcd73e35b8200e67" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 46535 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7b99824f1c6744f29b3c2787495f631c" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 37991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1c2c5c12d01441d843f06f985ca1684" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 39661 } health_report { overall_health: UNKNOWN } } }
I20260811 19:01:30.065809 24586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:30.057530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56592 (local address 127.29.112.129:46535)
0811 19:01:30.057925 (+ 395us) server_negotiation.cc:206] Beginning negotiation
0811 19:01:30.057932 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:01:30.057958 (+ 26us) server_negotiation.cc:407] Connection header received
0811 19:01:30.058096 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:01:30.058101 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:01:30.058211 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:01:30.058394 (+ 183us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:01:30.059016 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:30.060370 (+ 1354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:30.062692 (+ 2322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:30.063336 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:30.063803 (+ 467us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:30.064308 (+ 505us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:01:30.064937 (+ 629us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:01:30.065081 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 19:01:30.065379 (+ 298us) server_negotiation.cc:299] Negotiation successful
0811 19:01:30.065478 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159}
I20260811 19:01:30.075742 24486 raft_consensus.cc:1261] T 4f4db87d32b5453fa0f2636eaf926313 P d1c2c5c12d01441d843f06f985ca1684 [term 1 FOLLOWER]: Refusing update from remote peer 87495242ca044b7abcd73e35b8200e67: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:01:30.075804 24279 raft_consensus.cc:1261] T 4f4db87d32b5453fa0f2636eaf926313 P 7b99824f1c6744f29b3c2787495f631c [term 1 FOLLOWER]: Refusing update from remote peer 87495242ca044b7abcd73e35b8200e67: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:01:30.077123 24605 consensus_queue.cc:1035] T 4f4db87d32b5453fa0f2636eaf926313 P 87495242ca044b7abcd73e35b8200e67 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7b99824f1c6744f29b3c2787495f631c" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 37991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:01:30.078020 24598 consensus_queue.cc:1035] T 4f4db87d32b5453fa0f2636eaf926313 P 87495242ca044b7abcd73e35b8200e67 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1c2c5c12d01441d843f06f985ca1684" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 39661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:01:30.092155 24610 mvcc.cc:205] Tried to move back new op lower bound from 7317401149737406464 to 7317401149544910848. Current Snapshot: MvccSnapshot[applied={T|T < 7317401149737406464}]
I20260811 19:01:30.109959 23859 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36808:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:01:30.110337 23859 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36808:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:01:30.115592 23859 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 394f0e9f175e4f24b7e1081aaeb6fb3b: Sending DeleteTablet for 3 replicas of tablet 4f4db87d32b5453fa0f2636eaf926313
I20260811 19:01:30.117224 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 23943
I20260811 19:01:30.117393 24242 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f4db87d32b5453fa0f2636eaf926313 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:01:30 UTC) from {username='kudu', principal='kudu/127.29.112.190@KRBTEST.COM'} at 127.0.0.1:49952
I20260811 19:01:30.117815 24455 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f4db87d32b5453fa0f2636eaf926313 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:01:30 UTC) from {username='kudu', principal='kudu/127.29.112.190@KRBTEST.COM'} at 127.0.0.1:46814
I20260811 19:01:30.118142 24041 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f4db87d32b5453fa0f2636eaf926313 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:01:30 UTC) from {username='kudu', principal='kudu/127.29.112.190@KRBTEST.COM'} at 127.0.0.1:56580
I20260811 19:01:30.144081 24624 tablet_replica.cc:330] T 4f4db87d32b5453fa0f2636eaf926313 P d1c2c5c12d01441d843f06f985ca1684: stopping tablet replica
I20260811 19:01:30.144196 24627 tablet_replica.cc:330] T 4f4db87d32b5453fa0f2636eaf926313 P 87495242ca044b7abcd73e35b8200e67: stopping tablet replica
I20260811 19:01:30.144729 24623 tablet_replica.cc:330] T 4f4db87d32b5453fa0f2636eaf926313 P 7b99824f1c6744f29b3c2787495f631c: stopping tablet replica
I20260811 19:01:30.144989 24624 raft_consensus.cc:2229] T 4f4db87d32b5453fa0f2636eaf926313 P d1c2c5c12d01441d843f06f985ca1684 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:01:30.145411 24627 raft_consensus.cc:2229] T 4f4db87d32b5453fa0f2636eaf926313 P 87495242ca044b7abcd73e35b8200e67 [term 1 LEADER]: Raft consensus shutting down.
I20260811 19:01:30.145491 24623 raft_consensus.cc:2229] T 4f4db87d32b5453fa0f2636eaf926313 P 7b99824f1c6744f29b3c2787495f631c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:01:30.145856 24624 raft_consensus.cc:2258] T 4f4db87d32b5453fa0f2636eaf926313 P d1c2c5c12d01441d843f06f985ca1684 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:01:30.146203 24623 raft_consensus.cc:2258] T 4f4db87d32b5453fa0f2636eaf926313 P 7b99824f1c6744f29b3c2787495f631c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:01:30.147048 24627 raft_consensus.cc:2258] T 4f4db87d32b5453fa0f2636eaf926313 P 87495242ca044b7abcd73e35b8200e67 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:01:30.149292 24623 ts_tablet_manager.cc:1905] T 4f4db87d32b5453fa0f2636eaf926313 P 7b99824f1c6744f29b3c2787495f631c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:01:30.149292 24624 ts_tablet_manager.cc:1905] T 4f4db87d32b5453fa0f2636eaf926313 P d1c2c5c12d01441d843f06f985ca1684: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:01:30.152369 24627 ts_tablet_manager.cc:1905] T 4f4db87d32b5453fa0f2636eaf926313 P 87495242ca044b7abcd73e35b8200e67: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 11 19:01:30 dist-test-slave-h6n0 krb5kdc[23787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474886, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.129@KRBTEST.COM
I20260811 19:01:30.158294 24623 ts_tablet_manager.cc:1918] T 4f4db87d32b5453fa0f2636eaf926313 P 7b99824f1c6744f29b3c2787495f631c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:01:30.158667 24623 log.cc:1196] T 4f4db87d32b5453fa0f2636eaf926313 P 7b99824f1c6744f29b3c2787495f631c: Deleting WAL directory at /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-1/wal/wals/4f4db87d32b5453fa0f2636eaf926313
I20260811 19:01:30.159379 24623 ts_tablet_manager.cc:1939] T 4f4db87d32b5453fa0f2636eaf926313 P 7b99824f1c6744f29b3c2787495f631c: Deleting consensus metadata
I20260811 19:01:30.161787 23836 catalog_manager.cc:4841] TS 7b99824f1c6744f29b3c2787495f631c (127.29.112.130:37991): tablet 4f4db87d32b5453fa0f2636eaf926313 (table test-table [id=95ac80e3a5114483988226925e552289]) successfully deleted
I20260811 19:01:30.162881 24624 ts_tablet_manager.cc:1918] T 4f4db87d32b5453fa0f2636eaf926313 P d1c2c5c12d01441d843f06f985ca1684: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:01:30.163077 24624 log.cc:1196] T 4f4db87d32b5453fa0f2636eaf926313 P d1c2c5c12d01441d843f06f985ca1684: Deleting WAL directory at /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-2/wal/wals/4f4db87d32b5453fa0f2636eaf926313
I20260811 19:01:30.163663 24624 ts_tablet_manager.cc:1939] T 4f4db87d32b5453fa0f2636eaf926313 P d1c2c5c12d01441d843f06f985ca1684: Deleting consensus metadata
I20260811 19:01:30.165611 24586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:30.120089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56604 (local address 127.29.112.129:46535)
0811 19:01:30.120358 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0811 19:01:30.120368 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 19:01:30.142151 (+ 21783us) server_negotiation.cc:407] Connection header received
0811 19:01:30.144127 (+ 1976us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:01:30.144137 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:01:30.144247 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:01:30.144320 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:01:30.146715 (+ 2395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:30.147995 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:30.150142 (+ 2147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:30.150916 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:30.151032 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:30.157326 (+ 6294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:01:30.157344 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:01:30.157363 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:01:30.157407 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:01:30.161700 (+ 4293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:01:30.162735 (+ 1035us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:01:30.162745 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:01:30.162749 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:01:30.162944 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:01:30.163456 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:01:30.163462 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:01:30.163465 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:01:30.163955 (+ 490us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:01:30.164298 (+ 343us) server_negotiation.cc:1035] Waiting for connection context
0811 19:01:30.164928 (+ 630us) server_negotiation.cc:299] Negotiation successful
0811 19:01:30.165213 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260811 19:01:30.165660 23836 catalog_manager.cc:4841] TS d1c2c5c12d01441d843f06f985ca1684 (127.29.112.131:39661): tablet 4f4db87d32b5453fa0f2636eaf926313 (table test-table [id=95ac80e3a5114483988226925e552289]) successfully deleted
I20260811 19:01:30.167402 24131 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:01:30.273607 24627 ts_tablet_manager.cc:1918] T 4f4db87d32b5453fa0f2636eaf926313 P 87495242ca044b7abcd73e35b8200e67: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:01:30.274412 24627 log.cc:1196] T 4f4db87d32b5453fa0f2636eaf926313 P 87495242ca044b7abcd73e35b8200e67: Deleting WAL directory at /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-0/wal/wals/4f4db87d32b5453fa0f2636eaf926313
I20260811 19:01:30.274459 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 23943
I20260811 19:01:30.302500 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 24146
W20260811 19:01:30.302577 23835 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.112.129:46535 (error 108)
I20260811 19:01:30.304689 24589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:30.304073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:46535 (local address 127.0.0.1:56606)
0811 19:01:30.304313 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:30.304483 (+ 170us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.129:46535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
W20260811 19:01:30.305106 23835 catalog_manager.cc:4584] TS 87495242ca044b7abcd73e35b8200e67 (127.29.112.129:46535): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4f4db87d32b5453fa0f2636eaf926313: Network error: Client connection negotiation failed: client connection to 127.29.112.129:46535: connect: Connection refused (error 111)
Aug 11 19:01:30 dist-test-slave-h6n0 krb5kdc[23787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474886, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.130@KRBTEST.COM
I20260811 19:01:30.322819 24588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:30.303586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49962 (local address 127.29.112.130:37991)
0811 19:01:30.303814 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0811 19:01:30.303822 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:01:30.303938 (+ 116us) server_negotiation.cc:407] Connection header received
0811 19:01:30.304092 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:01:30.304101 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:01:30.304187 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:01:30.304296 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:01:30.306031 (+ 1735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:30.307328 (+ 1297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:30.309316 (+ 1988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:30.309792 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:30.309921 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:30.315279 (+ 5358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:01:30.315296 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:01:30.315344 (+ 48us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:01:30.315392 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:01:30.319033 (+ 3641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:01:30.320063 (+ 1030us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:01:30.320073 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:01:30.320075 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:01:30.320255 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:01:30.320765 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:01:30.320770 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:01:30.320773 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:01:30.321456 (+ 683us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:01:30.321782 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0811 19:01:30.322176 (+ 394us) server_negotiation.cc:299] Negotiation successful
0811 19:01:30.322390 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 19:01:30.323606 24342 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:01:30.359009 24589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:30.358430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:46535 (local address 127.0.0.1:56618)
0811 19:01:30.358744 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:30.358845 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.129:46535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136}
I20260811 19:01:30.360481 24589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:30.360064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:46535 (local address 127.0.0.1:56624)
0811 19:01:30.360241 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:30.360352 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.129:46535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260811 19:01:30.421630 24589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:30.421072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:46535 (local address 127.0.0.1:56638)
0811 19:01:30.421344 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:30.421436 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.129:46535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
I20260811 19:01:30.424453 24589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:30.423626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:46535 (local address 127.0.0.1:56640)
0811 19:01:30.423878 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:30.424128 (+ 250us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.129:46535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
I20260811 19:01:30.424494 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 24146
I20260811 19:01:30.457597 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 24358
Aug 11 19:01:30 dist-test-slave-h6n0 krb5kdc[23787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474886, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.131@KRBTEST.COM
I20260811 19:01:30.478391 24591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:30.458827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46818 (local address 127.29.112.131:39661)
0811 19:01:30.459063 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0811 19:01:30.459073 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 19:01:30.459097 (+ 24us) server_negotiation.cc:407] Connection header received
0811 19:01:30.459317 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:01:30.459366 (+ 49us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:01:30.459452 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:01:30.459573 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:01:30.461236 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:30.462153 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:30.464125 (+ 1972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:30.464778 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:30.464985 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:30.470266 (+ 5281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:01:30.470285 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:01:30.470302 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:01:30.470341 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:01:30.474158 (+ 3817us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:01:30.475539 (+ 1381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:01:30.475547 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:01:30.475550 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:01:30.475727 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:01:30.476187 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:01:30.476191 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:01:30.476193 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:01:30.476606 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:01:30.476921 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0811 19:01:30.477616 (+ 695us) server_negotiation.cc:299] Negotiation successful
0811 19:01:30.477881 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260811 19:01:30.479290 24558 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:01:30.506887 24589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:30.505938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:46535 (local address 127.0.0.1:56650)
0811 19:01:30.506324 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:30.506469 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.129:46535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188}
I20260811 19:01:30.508448 24589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:30.507898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:46535 (local address 127.0.0.1:56660)
0811 19:01:30.508220 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:30.508314 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.129:46535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207}
I20260811 19:01:30.588173 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 24358
I20260811 19:01:30.615681 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 23808
I20260811 19:01:30.616920 23901 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:01:30.739135 24589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:30.738405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:46535 (local address 127.0.0.1:56672)
0811 19:01:30.738776 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:30.738896 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.129:46535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203}
I20260811 19:01:30.739236 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 23808
I20260811 19:01:50.775807 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.190:45339
--webserver_interface=127.29.112.190
--webserver_port=43159
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab
--principal=kudu/127.29.112.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:36075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.190:45339
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:01:51.230957 24638 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:01:51.231384 24638 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:01:51.231568 24638 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:01:51.242395 24638 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 19:01:51.242673 24638 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 19:01:51.242779 24638 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 19:01:51.242890 24638 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:01:51.243011 24638 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:01:51.243209 24638 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:01:51.243315 24638 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:01:51.243413 24638 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:01:51.257997 24638 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:36075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.29.112.190:45339
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab
--principal=kudu/127.29.112.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.190:45339
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.112.190
--webserver_port=43159
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:01:51.259935 24638 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:01:51.261415 24638 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:01:51.296926 24649 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:01:51.296949 24651 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:01:51.296957 24647 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:01:51.298602 24638 server_base.cc:1034] running on GCE node
I20260811 19:01:51.299728 24638 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:01:51.301609 24638 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:01:51.302894 24638 hybrid_clock.cc:648] HybridClock initialized: now 1786474911302806 us; error 62 us; skew 500 ppm
I20260811 19:01:51.303450 24638 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:01:51 dist-test-slave-h6n0 krb5kdc[23787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474911, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:01:51.314347 24638 init.cc:377] Logged in from keytab as kudu/127.29.112.190@KRBTEST.COM (short username kudu)
I20260811 19:01:51.317457 24638 webserver.cc:460] Webserver started at http://127.29.112.190:43159/ using document root <none> and password file <none>
I20260811 19:01:51.318889 24638 fs_manager.cc:359] Metadata directory not provided
I20260811 19:01:51.319046 24638 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:01:51.326493 24638 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.002s sys 0.004s
I20260811 19:01:51.331547 24661 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:01:51.334651 24638 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.004s sys 0.002s
I20260811 19:01:51.335019 24638 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/master-0/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/master-0/wal
uuid: "394f0e9f175e4f24b7e1081aaeb6fb3b"
format_stamp: "Formatted at 2026-08-11 19:01:27 on dist-test-slave-h6n0"
server_key: "54b1f8faf9111f2625fb7e8489539574"
server_key_iv: "69b659430a273751b5130371a7442b26"
server_key_version: "encryptionkey@0"
I20260811 19:01:51.337126 24638 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:01:51.377272 24638 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:01:51.383348 24638 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:01:51.383737 24638 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:01:51.410562 24638 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.190:45339
I20260811 19:01:51.410910 24740 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.190:45339 every 8 connection(s)
I20260811 19:01:51.412741 24638 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
I20260811 19:01:51.414183 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 24638
I20260811 19:01:51.415251 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.129:46535
--local_ip_for_outbound_sockets=127.29.112.129
--tserver_master_addrs=127.29.112.190:45339
--webserver_port=45245
--webserver_interface=127.29.112.129
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.129.keytab
--principal=kudu/127.29.112.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:36075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 19:01:51.423178 24741 sys_catalog.cc:263] Verifying existing consensus state
I20260811 19:01:51.427484 24741 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 394f0e9f175e4f24b7e1081aaeb6fb3b: Bootstrap starting.
I20260811 19:01:51.467406 24741 log.cc:824] T 00000000000000000000000000000000 P 394f0e9f175e4f24b7e1081aaeb6fb3b: Log is configured to *not* fsync() on all Append() calls
I20260811 19:01:51.492995 24741 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 394f0e9f175e4f24b7e1081aaeb6fb3b: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 19:01:51.498744 24741 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 394f0e9f175e4f24b7e1081aaeb6fb3b: Bootstrap complete.
I20260811 19:01:51.520079 24741 raft_consensus.cc:348] T 00000000000000000000000000000000 P 394f0e9f175e4f24b7e1081aaeb6fb3b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "394f0e9f175e4f24b7e1081aaeb6fb3b" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 45339 } }
I20260811 19:01:51.521768 24741 raft_consensus.cc:729] T 00000000000000000000000000000000 P 394f0e9f175e4f24b7e1081aaeb6fb3b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 394f0e9f175e4f24b7e1081aaeb6fb3b, State: Initialized, Role: FOLLOWER
I20260811 19:01:51.522753 24741 consensus_queue.cc:260] T 00000000000000000000000000000000 P 394f0e9f175e4f24b7e1081aaeb6fb3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "394f0e9f175e4f24b7e1081aaeb6fb3b" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 45339 } }
I20260811 19:01:51.523115 24741 raft_consensus.cc:388] T 00000000000000000000000000000000 P 394f0e9f175e4f24b7e1081aaeb6fb3b [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:01:51.523276 24741 raft_consensus.cc:482] T 00000000000000000000000000000000 P 394f0e9f175e4f24b7e1081aaeb6fb3b [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:01:51.523538 24741 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 394f0e9f175e4f24b7e1081aaeb6fb3b [term 1 FOLLOWER]: Advancing to term 2
I20260811 19:01:51.532173 24741 raft_consensus.cc:504] T 00000000000000000000000000000000 P 394f0e9f175e4f24b7e1081aaeb6fb3b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "394f0e9f175e4f24b7e1081aaeb6fb3b" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 45339 } }
I20260811 19:01:51.532930 24741 leader_election.cc:302] T 00000000000000000000000000000000 P 394f0e9f175e4f24b7e1081aaeb6fb3b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 394f0e9f175e4f24b7e1081aaeb6fb3b; no voters:
I20260811 19:01:51.533835 24741 leader_election.cc:288] T 00000000000000000000000000000000 P 394f0e9f175e4f24b7e1081aaeb6fb3b [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 19:01:51.534184 24745 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 394f0e9f175e4f24b7e1081aaeb6fb3b [term 2 FOLLOWER]: Leader election won for term 2
I20260811 19:01:51.536046 24745 raft_consensus.cc:686] T 00000000000000000000000000000000 P 394f0e9f175e4f24b7e1081aaeb6fb3b [term 2 LEADER]: Becoming Leader. State: Replica: 394f0e9f175e4f24b7e1081aaeb6fb3b, State: Running, Role: LEADER
I20260811 19:01:51.536895 24741 sys_catalog.cc:564] T 00000000000000000000000000000000 P 394f0e9f175e4f24b7e1081aaeb6fb3b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:01:51.537065 24745 consensus_queue.cc:237] T 00000000000000000000000000000000 P 394f0e9f175e4f24b7e1081aaeb6fb3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "394f0e9f175e4f24b7e1081aaeb6fb3b" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 45339 } }
I20260811 19:01:51.541102 24746 sys_catalog.cc:455] T 00000000000000000000000000000000 P 394f0e9f175e4f24b7e1081aaeb6fb3b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "394f0e9f175e4f24b7e1081aaeb6fb3b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "394f0e9f175e4f24b7e1081aaeb6fb3b" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 45339 } } }
I20260811 19:01:51.541455 24748 sys_catalog.cc:455] T 00000000000000000000000000000000 P 394f0e9f175e4f24b7e1081aaeb6fb3b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 394f0e9f175e4f24b7e1081aaeb6fb3b. Latest consensus state: current_term: 2 leader_uuid: "394f0e9f175e4f24b7e1081aaeb6fb3b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "394f0e9f175e4f24b7e1081aaeb6fb3b" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 45339 } } }
I20260811 19:01:51.547917 24746 sys_catalog.cc:458] T 00000000000000000000000000000000 P 394f0e9f175e4f24b7e1081aaeb6fb3b [sys.catalog]: This master's current role is: LEADER
I20260811 19:01:51.548241 24748 sys_catalog.cc:458] T 00000000000000000000000000000000 P 394f0e9f175e4f24b7e1081aaeb6fb3b [sys.catalog]: This master's current role is: LEADER
I20260811 19:01:51.549952 24757 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:01:51.565056 24757 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:01:51.566850 24757 catalog_manager.cc:1259] Loaded cluster ID: 79b877f1f5ac49f4907b1d7cb27744fa
I20260811 19:01:51.567085 24757 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:01:51.573087 24757 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:01:51.575671 24757 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 394f0e9f175e4f24b7e1081aaeb6fb3b: Loaded TSK: 0
I20260811 19:01:51.619726 24757 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 394f0e9f175e4f24b7e1081aaeb6fb3b: Generated new TSK 1
I20260811 19:01:51.620224 24757 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 19:01:51.943457 24743 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:01:51.943871 24743 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:01:51.944047 24743 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:01:51.956298 24743 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:01:51.956521 24743 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:01:51.956897 24743 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.129
I20260811 19:01:51.970484 24743 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:36075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.129.keytab
--principal=kudu/127.29.112.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.129:46535
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.112.129
--webserver_port=45245
--tserver_master_addrs=127.29.112.190:45339
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.129
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:01:51.972287 24743 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:01:51.973727 24743 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:01:51.991770 24782 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:01:51.992479 24784 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:01:51.992700 24781 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:01:51.993402 24743 server_base.cc:1034] running on GCE node
I20260811 19:01:51.994472 24743 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:01:51.996420 24743 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:01:51.997888 24743 hybrid_clock.cc:648] HybridClock initialized: now 1786474911997780 us; error 88 us; skew 500 ppm
I20260811 19:01:51.998457 24743 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:01:51 dist-test-slave-h6n0 krb5kdc[23787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474911, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:01:52.002426 24743 init.cc:377] Logged in from keytab as kudu/127.29.112.129@KRBTEST.COM (short username kudu)
I20260811 19:01:52.004498 24743 webserver.cc:460] Webserver started at http://127.29.112.129:45245/ using document root <none> and password file <none>
I20260811 19:01:52.005494 24743 fs_manager.cc:359] Metadata directory not provided
I20260811 19:01:52.005712 24743 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:01:52.013231 24743 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 19:01:52.016561 24793 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:01:52.019007 24743 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260811 19:01:52.019217 24743 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-0/wal
uuid: "87495242ca044b7abcd73e35b8200e67"
format_stamp: "Formatted at 2026-08-11 19:01:28 on dist-test-slave-h6n0"
server_key: "e3e7ca1842fbc341318292979be8d2b0"
server_key_iv: "778c5330775e38609965abf788fc5b59"
server_key_version: "encryptionkey@0"
I20260811 19:01:52.020283 24743 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:01:52.046124 24743 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:01:52.052213 24743 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:01:52.052561 24743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:01:52.053781 24743 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:01:52.056178 24801 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
W20260811 19:01:52.062719 24801 ts_tablet_manager.cc:1852] T 4f4db87d32b5453fa0f2636eaf926313 P 87495242ca044b7abcd73e35b8200e67: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260811 19:01:52.064392 24801 ts_tablet_manager.cc:1905] T 4f4db87d32b5453fa0f2636eaf926313 P 87495242ca044b7abcd73e35b8200e67: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:01:52.074077 24801 ts_tablet_manager.cc:1918] T 4f4db87d32b5453fa0f2636eaf926313 P 87495242ca044b7abcd73e35b8200e67: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:01:52.074306 24801 log.cc:1196] T 4f4db87d32b5453fa0f2636eaf926313 P 87495242ca044b7abcd73e35b8200e67: Deleting WAL directory at /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-0/wal/wals/4f4db87d32b5453fa0f2636eaf926313
I20260811 19:01:52.074893 24801 ts_tablet_manager.cc:1939] T 4f4db87d32b5453fa0f2636eaf926313 P 87495242ca044b7abcd73e35b8200e67: Deleting consensus metadata
I20260811 19:01:52.075378 24743 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260811 19:01:52.075490 24743 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.020s user 0.001s sys 0.000s
I20260811 19:01:52.075593 24743 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:01:52.075663 24743 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:01:52.120850 24743 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.129:46535
I20260811 19:01:52.121035 24963 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.129:46535 every 8 connection(s)
I20260811 19:01:52.122458 24743 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-0/data/info.pb
I20260811 19:01:52.131371 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 24743
I20260811 19:01:52.132509 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.130:37991
--local_ip_for_outbound_sockets=127.29.112.130
--tserver_master_addrs=127.29.112.190:45339
--webserver_port=33725
--webserver_interface=127.29.112.130
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.130.keytab
--principal=kudu/127.29.112.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:36075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 19:01:52 dist-test-slave-h6n0 krb5kdc[23787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474911, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.129@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 19:01:52.156294 24973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:52.126067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.129:60823 (local address 127.29.112.190:45339)
0811 19:01:52.127091 (+ 1024us) server_negotiation.cc:206] Beginning negotiation
0811 19:01:52.127116 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0811 19:01:52.127314 (+ 198us) server_negotiation.cc:407] Connection header received
0811 19:01:52.130033 (+ 2719us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:01:52.130059 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:01:52.131695 (+ 1636us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:01:52.132181 (+ 486us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:01:52.134007 (+ 1826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:52.136379 (+ 2372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:52.139218 (+ 2839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:52.140575 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:52.140839 (+ 264us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:52.145468 (+ 4629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:01:52.145616 (+ 148us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:01:52.145661 (+ 45us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:01:52.145750 (+ 89us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:01:52.150892 (+ 5142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:01:52.152011 (+ 1119us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:01:52.152019 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:01:52.152024 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:01:52.152199 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:01:52.152698 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:01:52.152703 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:01:52.152705 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:01:52.153251 (+ 546us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:01:52.153497 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0811 19:01:52.154367 (+ 870us) server_negotiation.cc:299] Negotiation successful
0811 19:01:52.154822 (+ 455us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":743,"thread_start_us":450,"threads_started":1}
I20260811 19:01:52.156352 24969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:52.126131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:45339 (local address 127.29.112.129:60823)
0811 19:01:52.127079 (+ 948us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:52.127130 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 19:01:52.129151 (+ 2021us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:01:52.132349 (+ 3198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:01:52.132357 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:01:52.133127 (+ 770us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:01:52.133726 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:01:52.133745 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:01:52.136759 (+ 3014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:01:52.136769 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:01:52.138932 (+ 2163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:01:52.138977 (+ 45us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:01:52.141196 (+ 2219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:01:52.141203 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:01:52.141454 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:52.142311 (+ 857us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:01:52.142338 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:01:52.145058 (+ 2720us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:01:52.151101 (+ 6043us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:01:52.151120 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:01:52.151134 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:01:52.151819 (+ 685us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:01:52.152361 (+ 542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:01:52.152366 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:01:52.152368 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:01:52.152552 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:01:52.153427 (+ 875us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:01:52.153438 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:01:52.153819 (+ 381us) client_negotiation.cc:769] Sending connection context
0811 19:01:52.154120 (+ 301us) client_negotiation.cc:240] Negotiation successful
0811 19:01:52.154797 (+ 677us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":624,"thread_start_us":416,"threads_started":1}
I20260811 19:01:52.159508 24965 heartbeater.cc:347] Connected to a master server at 127.29.112.190:45339
I20260811 19:01:52.159880 24965 heartbeater.cc:464] Registering TS with master...
I20260811 19:01:52.160769 24965 heartbeater.cc:511] Master 127.29.112.190:45339 requested a full tablet report, sending...
I20260811 19:01:52.163322 24685 ts_manager.cc:194] Registered new tserver with Master: 87495242ca044b7abcd73e35b8200e67 (127.29.112.129:46535)
I20260811 19:01:52.165174 24685 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.129@KRBTEST.COM'} at 127.29.112.129:60823
W20260811 19:01:52.558589 24975 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:01:52.558954 24975 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:01:52.559134 24975 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:01:52.567193 24975 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:01:52.567376 24975 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:01:52.567574 24975 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.130
I20260811 19:01:52.578317 24975 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:36075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.130.keytab
--principal=kudu/127.29.112.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.130:37991
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.112.130
--webserver_port=33725
--tserver_master_addrs=127.29.112.190:45339
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.130
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:01:52.579782 24975 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:01:52.581209 24975 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:01:52.596249 24987 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:01:52.597054 24990 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:01:52.597072 24988 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:01:52.597261 24975 server_base.cc:1034] running on GCE node
I20260811 19:01:52.598232 24975 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:01:52.599399 24975 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:01:52.600780 24975 hybrid_clock.cc:648] HybridClock initialized: now 1786474912600681 us; error 42 us; skew 500 ppm
I20260811 19:01:52.601604 24975 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:01:52 dist-test-slave-h6n0 krb5kdc[23787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474912, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:01:52.605835 24975 init.cc:377] Logged in from keytab as kudu/127.29.112.130@KRBTEST.COM (short username kudu)
I20260811 19:01:52.607864 24975 webserver.cc:460] Webserver started at http://127.29.112.130:33725/ using document root <none> and password file <none>
I20260811 19:01:52.608619 24975 fs_manager.cc:359] Metadata directory not provided
I20260811 19:01:52.608794 24975 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:01:52.614596 24975 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260811 19:01:52.617928 25000 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:01:52.619830 24975 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 19:01:52.620054 24975 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-1/wal
uuid: "7b99824f1c6744f29b3c2787495f631c"
format_stamp: "Formatted at 2026-08-11 19:01:28 on dist-test-slave-h6n0"
server_key: "5d7517d24d4f31e365c04e607ea07445"
server_key_iv: "205d31e6af17ae27cd13e7d9574e79cc"
server_key_version: "encryptionkey@0"
I20260811 19:01:52.621222 24975 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:01:52.635639 24975 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:01:52.640785 24975 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:01:52.641098 24975 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:01:52.642494 24975 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:01:52.644447 24975 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:01:52.644598 24975 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:01:52.644745 24975 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:01:52.644868 24975 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:01:52.691228 24975 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.130:37991
I20260811 19:01:52.691298 25177 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.130:37991 every 8 connection(s)
I20260811 19:01:52.693678 24975 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-1/data/info.pb
I20260811 19:01:52.697969 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 24975
I20260811 19:01:52.698967 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.131:39661
--local_ip_for_outbound_sockets=127.29.112.131
--tserver_master_addrs=127.29.112.190:45339
--webserver_port=41429
--webserver_interface=127.29.112.131
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.131.keytab
--principal=kudu/127.29.112.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:36075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 19:01:52 dist-test-slave-h6n0 krb5kdc[23787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474912, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.130@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 19:01:52.726176 25181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:52.698322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.130:33319 (local address 127.29.112.190:45339)
0811 19:01:52.699136 (+ 814us) server_negotiation.cc:206] Beginning negotiation
0811 19:01:52.699146 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 19:01:52.700829 (+ 1683us) server_negotiation.cc:407] Connection header received
0811 19:01:52.704464 (+ 3635us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:01:52.704473 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:01:52.704589 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:01:52.704753 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:01:52.707682 (+ 2929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:52.708596 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:52.711869 (+ 3273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:52.712320 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:52.712435 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:52.716790 (+ 4355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:01:52.716800 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:01:52.716808 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:01:52.716864 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:01:52.721568 (+ 4704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:01:52.722428 (+ 860us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:01:52.722437 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:01:52.722444 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:01:52.722659 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:01:52.723179 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:01:52.723187 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:01:52.723194 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:01:52.723642 (+ 448us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:01:52.723849 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0811 19:01:52.725324 (+ 1475us) server_negotiation.cc:299] Negotiation successful
0811 19:01:52.725626 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":623,"thread_start_us":366,"threads_started":1}
I20260811 19:01:52.727875 25182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:52.699146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:45339 (local address 127.29.112.130:33319)
0811 19:01:52.700567 (+ 1421us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:52.700641 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0811 19:01:52.703873 (+ 3232us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:01:52.705363 (+ 1490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:01:52.705379 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:01:52.706372 (+ 993us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:01:52.707364 (+ 992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:01:52.707398 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:01:52.708802 (+ 1404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:01:52.708810 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:01:52.711657 (+ 2847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:01:52.711679 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:01:52.712592 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:01:52.712603 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:01:52.712981 (+ 378us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:52.713906 (+ 925us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:01:52.713938 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:01:52.716480 (+ 2542us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:01:52.721746 (+ 5266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:01:52.721755 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:01:52.721768 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:01:52.722254 (+ 486us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:01:52.722796 (+ 542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:01:52.722800 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:01:52.722803 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:01:52.723035 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:01:52.723947 (+ 912us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:01:52.723958 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:01:52.724750 (+ 792us) client_negotiation.cc:769] Sending connection context
0811 19:01:52.725130 (+ 380us) client_negotiation.cc:240] Negotiation successful
0811 19:01:52.725983 (+ 853us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":755,"thread_start_us":395,"threads_started":1}
I20260811 19:01:52.729563 25179 heartbeater.cc:347] Connected to a master server at 127.29.112.190:45339
I20260811 19:01:52.729964 25179 heartbeater.cc:464] Registering TS with master...
I20260811 19:01:52.731043 25179 heartbeater.cc:511] Master 127.29.112.190:45339 requested a full tablet report, sending...
I20260811 19:01:52.732981 24685 ts_manager.cc:194] Registered new tserver with Master: 7b99824f1c6744f29b3c2787495f631c (127.29.112.130:37991)
I20260811 19:01:52.733928 24685 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.130@KRBTEST.COM'} at 127.29.112.130:33319
W20260811 19:01:53.125398 25184 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:01:53.125825 25184 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:01:53.126055 25184 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:01:53.136377 25184 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:01:53.136576 25184 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:01:53.136806 25184 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.131
I20260811 19:01:53.148903 25184 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:36075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.131.keytab
--principal=kudu/127.29.112.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.131:39661
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.112.131
--webserver_port=41429
--tserver_master_addrs=127.29.112.190:45339
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.131
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:01:53.150309 25184 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:01:53.151856 25184 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 19:01:53.169090 24965 heartbeater.cc:503] Master 127.29.112.190:45339 was elected leader, sending a full tablet report...
W20260811 19:01:53.169605 25194 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:01:53.169580 25195 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:01:53.171144 25184 server_base.cc:1034] running on GCE node
W20260811 19:01:53.171233 25197 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:01:53.172170 25184 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:01:53.173390 25184 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:01:53.174680 25184 hybrid_clock.cc:648] HybridClock initialized: now 1786474913174589 us; error 60 us; skew 500 ppm
I20260811 19:01:53.175087 25184 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:01:53 dist-test-slave-h6n0 krb5kdc[23787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474913, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:01:53.179718 25184 init.cc:377] Logged in from keytab as kudu/127.29.112.131@KRBTEST.COM (short username kudu)
I20260811 19:01:53.181798 25184 webserver.cc:460] Webserver started at http://127.29.112.131:41429/ using document root <none> and password file <none>
I20260811 19:01:53.182670 25184 fs_manager.cc:359] Metadata directory not provided
I20260811 19:01:53.182837 25184 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:01:53.188480 25184 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 19:01:53.191725 25208 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:01:53.193145 25184 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 19:01:53.193360 25184 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-2/wal
uuid: "d1c2c5c12d01441d843f06f985ca1684"
format_stamp: "Formatted at 2026-08-11 19:01:29 on dist-test-slave-h6n0"
server_key: "391405d26b86085610c85f500acb08b8"
server_key_iv: "35bf95b36cd7cb7ba01aefd9acfcf118"
server_key_version: "encryptionkey@0"
I20260811 19:01:53.194398 25184 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:01:53.257871 25184 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:01:53.264851 25184 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:01:53.265383 25184 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:01:53.266755 25184 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:01:53.268682 25184 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:01:53.268870 25184 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:01:53.269030 25184 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:01:53.269146 25184 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:01:53.316502 25184 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.131:39661
I20260811 19:01:53.316660 25390 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.131:39661 every 8 connection(s)
I20260811 19:01:53.318153 25184 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-2/data/info.pb
I20260811 19:01:53.326742 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 25184
Aug 11 19:01:53 dist-test-slave-h6n0 krb5kdc[23787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474913, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.131@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 19:01:53.344830 25394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:53.321526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.131:36461 (local address 127.29.112.190:45339)
0811 19:01:53.322060 (+ 534us) server_negotiation.cc:206] Beginning negotiation
0811 19:01:53.322067 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:01:53.323100 (+ 1033us) server_negotiation.cc:407] Connection header received
0811 19:01:53.325121 (+ 2021us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:01:53.325127 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:01:53.325225 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:01:53.325374 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:01:53.327674 (+ 2300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:53.329207 (+ 1533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:53.330878 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:53.331446 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:53.331573 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:53.336678 (+ 5105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:01:53.336685 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:01:53.336689 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:01:53.336749 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:01:53.340677 (+ 3928us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:01:53.341581 (+ 904us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:01:53.341591 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:01:53.341593 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:01:53.341730 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:01:53.342204 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:01:53.342208 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:01:53.342210 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:01:53.342518 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:01:53.342778 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0811 19:01:53.344076 (+ 1298us) server_negotiation.cc:299] Negotiation successful
0811 19:01:53.344414 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":424,"thread_start_us":279,"threads_started":1}
I20260811 19:01:53.345845 25398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:53.321752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:45339 (local address 127.29.112.131:36461)
0811 19:01:53.322778 (+ 1026us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:53.322843 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0811 19:01:53.324813 (+ 1970us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:01:53.325723 (+ 910us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:01:53.325731 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:01:53.326364 (+ 633us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:01:53.327337 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:01:53.327358 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:01:53.329385 (+ 2027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:01:53.329390 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:01:53.330713 (+ 1323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:01:53.330726 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:01:53.331738 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:01:53.331747 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:01:53.332105 (+ 358us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:53.332943 (+ 838us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:01:53.332982 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:01:53.336303 (+ 3321us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:01:53.340868 (+ 4565us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:01:53.340879 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:01:53.340905 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:01:53.341438 (+ 533us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:01:53.341857 (+ 419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:01:53.341862 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:01:53.341866 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:01:53.342067 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:01:53.342716 (+ 649us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:01:53.342725 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:01:53.343456 (+ 731us) client_negotiation.cc:769] Sending connection context
0811 19:01:53.343814 (+ 358us) client_negotiation.cc:240] Negotiation successful
0811 19:01:53.344459 (+ 645us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":695,"thread_start_us":414,"threads_started":1}
I20260811 19:01:53.347618 25391 heartbeater.cc:347] Connected to a master server at 127.29.112.190:45339
I20260811 19:01:53.347968 25391 heartbeater.cc:464] Registering TS with master...
I20260811 19:01:53.348861 25399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:53.328630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33462 (local address 127.29.112.190:45339)
0811 19:01:53.329374 (+ 744us) server_negotiation.cc:206] Beginning negotiation
0811 19:01:53.329385 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 19:01:53.331183 (+ 1798us) server_negotiation.cc:407] Connection header received
0811 19:01:53.331340 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:01:53.331346 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:01:53.331449 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:01:53.331554 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:01:53.333944 (+ 2390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:53.335223 (+ 1279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:53.336931 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:53.337401 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:53.337556 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:53.342038 (+ 4482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:01:53.342046 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:01:53.342050 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:01:53.342189 (+ 139us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:01:53.345542 (+ 3353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:01:53.346460 (+ 918us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:01:53.346469 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:01:53.346476 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:01:53.346696 (+ 220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:01:53.347155 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:01:53.347160 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:01:53.347164 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:01:53.347475 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:01:53.347763 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0811 19:01:53.348186 (+ 423us) server_negotiation.cc:299] Negotiation successful
0811 19:01:53.348450 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":552,"thread_start_us":324,"threads_started":1}
I20260811 19:01:53.348891 25391 heartbeater.cc:511] Master 127.29.112.190:45339 requested a full tablet report, sending...
I20260811 19:01:53.351440 24685 ts_manager.cc:194] Registered new tserver with Master: d1c2c5c12d01441d843f06f985ca1684 (127.29.112.131:39661)
I20260811 19:01:53.353077 24685 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.131@KRBTEST.COM'} at 127.29.112.131:36461
I20260811 19:01:53.353575 30146 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 19:01:53.363211 25399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:53.355248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33470 (local address 127.29.112.190:45339)
0811 19:01:53.355617 (+ 369us) server_negotiation.cc:206] Beginning negotiation
0811 19:01:53.355626 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 19:01:53.355991 (+ 365us) server_negotiation.cc:407] Connection header received
0811 19:01:53.356618 (+ 627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:01:53.356624 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:01:53.356789 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:01:53.356995 (+ 206us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:01:53.357546 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:53.358334 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:53.360119 (+ 1785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:53.360657 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:53.360833 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:53.362174 (+ 1341us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:01:53.362468 (+ 294us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0811 19:01:53.362956 (+ 488us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:33470: token expired
Metrics: {"server-negotiator.queue_time_us":154}
W20260811 19:01:53.363339 25399 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:33470: token expired
W20260811 19:01:53.364778 24579 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.29.112.190:45339)
I20260811 19:01:53.385632 25399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:53.365653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33472 (local address 127.29.112.190:45339)
0811 19:01:53.366037 (+ 384us) server_negotiation.cc:206] Beginning negotiation
0811 19:01:53.366042 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:01:53.366382 (+ 340us) server_negotiation.cc:407] Connection header received
0811 19:01:53.366735 (+ 353us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:01:53.366741 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:01:53.366827 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:01:53.366934 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:01:53.369201 (+ 2267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:53.370044 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:53.372308 (+ 2264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:53.372841 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:53.373052 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:53.377222 (+ 4170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:01:53.377228 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:01:53.377232 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:01:53.377278 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:01:53.381275 (+ 3997us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:01:53.382481 (+ 1206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:01:53.382488 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:01:53.382491 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:01:53.382813 (+ 322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:01:53.383521 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:01:53.383527 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:01:53.383530 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:01:53.384010 (+ 480us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:01:53.384267 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0811 19:01:53.384976 (+ 709us) server_negotiation.cc:299] Negotiation successful
0811 19:01:53.385338 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":242}
I20260811 19:01:53.391606 24685 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33472:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 19:01:53.400878 24685 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 19:01:53.439356 25408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:53.417214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:46535 (local address 127.0.0.1:60310)
0811 19:01:53.418335 (+ 1121us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:53.418407 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0811 19:01:53.418676 (+ 269us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:01:53.421692 (+ 3016us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:01:53.421703 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:01:53.421749 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:01:53.422369 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:01:53.422387 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:01:53.427730 (+ 5343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:01:53.427739 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:01:53.432025 (+ 4286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:01:53.432055 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:01:53.438057 (+ 6002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:01:53.438067 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:01:53.438321 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:53.438378 (+ 57us) client_negotiation.cc:769] Sending connection context
0811 19:01:53.438593 (+ 215us) client_negotiation.cc:240] Negotiation successful
0811 19:01:53.438853 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":779,"thread_start_us":609,"threads_started":1}
I20260811 19:01:53.439498 25414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:53.417957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60310 (local address 127.29.112.129:46535)
0811 19:01:53.420535 (+ 2578us) server_negotiation.cc:206] Beginning negotiation
0811 19:01:53.420548 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 19:01:53.420587 (+ 39us) server_negotiation.cc:407] Connection header received
0811 19:01:53.420719 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:01:53.420731 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:01:53.421079 (+ 348us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:01:53.421330 (+ 251us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:01:53.423726 (+ 2396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:53.425865 (+ 2139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:53.432304 (+ 6439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:53.437132 (+ 4828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:53.437367 (+ 235us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:53.437632 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0811 19:01:53.438713 (+ 1081us) server_negotiation.cc:299] Negotiation successful
0811 19:01:53.439004 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2361,"thread_start_us":2220,"threads_started":1}
I20260811 19:01:53.442268 25409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:53.417417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.130:37991 (local address 127.0.0.1:60790)
0811 19:01:53.423381 (+ 5964us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:53.423409 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0811 19:01:53.423533 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:01:53.427329 (+ 3796us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:01:53.427334 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:01:53.427367 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:01:53.428044 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:01:53.428084 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:01:53.435075 (+ 6991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:01:53.435084 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:01:53.437838 (+ 2754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:01:53.438069 (+ 231us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:01:53.441085 (+ 3016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:01:53.441094 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:01:53.441334 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:53.441385 (+ 51us) client_negotiation.cc:769] Sending connection context
0811 19:01:53.441494 (+ 109us) client_negotiation.cc:240] Negotiation successful
0811 19:01:53.441651 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5730,"thread_start_us":5499,"threads_started":1}
I20260811 19:01:53.443472 25413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:53.417911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60790 (local address 127.29.112.130:37991)
0811 19:01:53.426547 (+ 8636us) server_negotiation.cc:206] Beginning negotiation
0811 19:01:53.426560 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 19:01:53.426599 (+ 39us) server_negotiation.cc:407] Connection header received
0811 19:01:53.426726 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:01:53.426738 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:01:53.427079 (+ 341us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:01:53.427331 (+ 252us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:01:53.432651 (+ 5320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:53.434837 (+ 2186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:53.438608 (+ 3771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:53.440731 (+ 2123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:53.441004 (+ 273us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:53.441220 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0811 19:01:53.442717 (+ 1497us) server_negotiation.cc:299] Negotiation successful
0811 19:01:53.443070 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8371,"thread_start_us":8198,"threads_started":1}
I20260811 19:01:53.443431 24860 tablet_service.cc:1467] Processing CreateTablet for tablet fe0ceca7e3ec4e97a9a5ed9ea86d18c9 (DEFAULT_TABLE table=test-table [id=c8afcc1e9ddf4d3f99a2fcd0263d0595]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:01:53.445740 24860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe0ceca7e3ec4e97a9a5ed9ea86d18c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:01:53.447511 25416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:53.425101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.131:39661 (local address 127.0.0.1:36654)
0811 19:01:53.437092 (+ 11991us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:53.437117 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 19:01:53.437277 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:01:53.439122 (+ 1845us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:01:53.439127 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:01:53.439156 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:01:53.439593 (+ 437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:01:53.439606 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:01:53.442041 (+ 2435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:01:53.442048 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:01:53.444536 (+ 2488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:01:53.444556 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:01:53.446397 (+ 1841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:01:53.446407 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:01:53.446642 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:53.446695 (+ 53us) client_negotiation.cc:769] Sending connection context
0811 19:01:53.446837 (+ 142us) client_negotiation.cc:240] Negotiation successful
0811 19:01:53.447000 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11814,"thread_start_us":11605,"threads_started":1}
I20260811 19:01:53.447507 25415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:53.423905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36654 (local address 127.29.112.131:39661)
0811 19:01:53.438007 (+ 14102us) server_negotiation.cc:206] Beginning negotiation
0811 19:01:53.438018 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 19:01:53.438058 (+ 40us) server_negotiation.cc:407] Connection header received
0811 19:01:53.438173 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:01:53.438182 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:01:53.438512 (+ 330us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:01:53.438774 (+ 262us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:01:53.439817 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:53.441734 (+ 1917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:53.444824 (+ 3090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:53.446141 (+ 1317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:53.446405 (+ 264us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:53.446642 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0811 19:01:53.446889 (+ 247us) server_negotiation.cc:299] Negotiation successful
0811 19:01:53.447075 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13845,"thread_start_us":13664,"threads_started":1}
I20260811 19:01:53.449056 25076 tablet_service.cc:1467] Processing CreateTablet for tablet fe0ceca7e3ec4e97a9a5ed9ea86d18c9 (DEFAULT_TABLE table=test-table [id=c8afcc1e9ddf4d3f99a2fcd0263d0595]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:01:53.451659 25076 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe0ceca7e3ec4e97a9a5ed9ea86d18c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:01:53.453904 25290 tablet_service.cc:1467] Processing CreateTablet for tablet fe0ceca7e3ec4e97a9a5ed9ea86d18c9 (DEFAULT_TABLE table=test-table [id=c8afcc1e9ddf4d3f99a2fcd0263d0595]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:01:53.455328 25290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe0ceca7e3ec4e97a9a5ed9ea86d18c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:01:53.460157 25417 tablet_bootstrap.cc:492] T fe0ceca7e3ec4e97a9a5ed9ea86d18c9 P 87495242ca044b7abcd73e35b8200e67: Bootstrap starting.
I20260811 19:01:53.464548 25417 tablet_bootstrap.cc:654] T fe0ceca7e3ec4e97a9a5ed9ea86d18c9 P 87495242ca044b7abcd73e35b8200e67: Neither blocks nor log segments found. Creating new log.
I20260811 19:01:53.466079 25417 log.cc:824] T fe0ceca7e3ec4e97a9a5ed9ea86d18c9 P 87495242ca044b7abcd73e35b8200e67: Log is configured to *not* fsync() on all Append() calls
I20260811 19:01:53.468667 25423 tablet_bootstrap.cc:492] T fe0ceca7e3ec4e97a9a5ed9ea86d18c9 P 7b99824f1c6744f29b3c2787495f631c: Bootstrap starting.
I20260811 19:01:53.469779 25417 tablet_bootstrap.cc:492] T fe0ceca7e3ec4e97a9a5ed9ea86d18c9 P 87495242ca044b7abcd73e35b8200e67: No bootstrap required, opened a new log
I20260811 19:01:53.470077 25417 ts_tablet_manager.cc:1397] T fe0ceca7e3ec4e97a9a5ed9ea86d18c9 P 87495242ca044b7abcd73e35b8200e67: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260811 19:01:53.471733 25425 tablet_bootstrap.cc:492] T fe0ceca7e3ec4e97a9a5ed9ea86d18c9 P d1c2c5c12d01441d843f06f985ca1684: Bootstrap starting.
I20260811 19:01:53.473860 25423 tablet_bootstrap.cc:654] T fe0ceca7e3ec4e97a9a5ed9ea86d18c9 P 7b99824f1c6744f29b3c2787495f631c: Neither blocks nor log segments found. Creating new log.
I20260811 19:01:53.475039 25425 tablet_bootstrap.cc:654] T fe0ceca7e3ec4e97a9a5ed9ea86d18c9 P d1c2c5c12d01441d843f06f985ca1684: Neither blocks nor log segments found. Creating new log.
I20260811 19:01:53.475340 25423 log.cc:824] T fe0ceca7e3ec4e97a9a5ed9ea86d18c9 P 7b99824f1c6744f29b3c2787495f631c: Log is configured to *not* fsync() on all Append() calls
I20260811 19:01:53.476572 25425 log.cc:824] T fe0ceca7e3ec4e97a9a5ed9ea86d18c9 P d1c2c5c12d01441d843f06f985ca1684: Log is configured to *not* fsync() on all Append() calls
I20260811 19:01:53.479574 25423 tablet_bootstrap.cc:492] T fe0ceca7e3ec4e97a9a5ed9ea86d18c9 P 7b99824f1c6744f29b3c2787495f631c: No bootstrap required, opened a new log
I20260811 19:01:53.479929 25423 ts_tablet_manager.cc:1397] T fe0ceca7e3ec4e97a9a5ed9ea86d18c9 P 7b99824f1c6744f29b3c2787495f631c: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260811 19:01:53.480418 25425 tablet_bootstrap.cc:492] T fe0ceca7e3ec4e97a9a5ed9ea86d18c9 P d1c2c5c12d01441d843f06f985ca1684: No bootstrap required, opened a new log
I20260811 19:01:53.480788 25425 ts_tablet_manager.cc:1397] T fe0ceca7e3ec4e97a9a5ed9ea86d18c9 P d1c2c5c12d01441d843f06f985ca1684: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260811 19:01:53.481732 25417 raft_consensus.cc:348] T fe0ceca7e3ec4e97a9a5ed9ea86d18c9 P 87495242ca044b7abcd73e35b8200e67 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87495242ca044b7abcd73e35b8200e67" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 46535 } } peers { permanent_uuid: "7b99824f1c6744f29b3c2787495f631c" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 37991 } } peers { permanent_uuid: "d1c2c5c12d01441d843f06f985ca1684" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 39661 } }
I20260811 19:01:53.482228 25417 raft_consensus.cc:374] T fe0ceca7e3ec4e97a9a5ed9ea86d18c9 P 87495242ca044b7abcd73e35b8200e67 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:01:53.482345 25417 raft_consensus.cc:729] T fe0ceca7e3ec4e97a9a5ed9ea86d18c9 P 87495242ca044b7abcd73e35b8200e67 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87495242ca044b7abcd73e35b8200e67, State: Initialized, Role: FOLLOWER
I20260811 19:01:53.483093 25417 consensus_queue.cc:260] T fe0ceca7e3ec4e97a9a5ed9ea86d18c9 P 87495242ca044b7abcd73e35b8200e67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87495242ca044b7abcd73e35b8200e67" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 46535 } } peers { permanent_uuid: "7b99824f1c6744f29b3c2787495f631c" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 37991 } } peers { permanent_uuid: "d1c2c5c12d01441d843f06f985ca1684" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 39661 } }
I20260811 19:01:53.485206 25417 ts_tablet_manager.cc:1428] T fe0ceca7e3ec4e97a9a5ed9ea86d18c9 P 87495242ca044b7abcd73e35b8200e67: Time spent starting tablet: real 0.015s user 0.016s sys 0.000s
I20260811 19:01:53.487423 25431 raft_consensus.cc:482] T fe0ceca7e3ec4e97a9a5ed9ea86d18c9 P 87495242ca044b7abcd73e35b8200e67 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:01:53.487773 25431 raft_consensus.cc:504] T fe0ceca7e3ec4e97a9a5ed9ea86d18c9 P 87495242ca044b7abcd73e35b8200e67 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87495242ca044b7abcd73e35b8200e67" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 46535 } } peers { permanent_uuid: "7b99824f1c6744f29b3c2787495f631c" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 37991 } } peers { permanent_uuid: "d1c2c5c12d01441d843f06f985ca1684" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 39661 } }
I20260811 19:01:53.489868 25431 leader_election.cc:288] T fe0ceca7e3ec4e97a9a5ed9ea86d18c9 P 87495242ca044b7abcd73e35b8200e67 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7b99824f1c6744f29b3c2787495f631c (127.29.112.130:37991), d1c2c5c12d01441d843f06f985ca1684 (127.29.112.131:39661)
I20260811 19:01:53.493044 25425 raft_consensus.cc:348] T fe0ceca7e3ec4e97a9a5ed9ea86d18c9 P d1c2c5c12d01441d843f06f985ca1684 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87495242ca044b7abcd73e35b8200e67" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 46535 } } peers { permanent_uuid: "7b99824f1c6744f29b3c2787495f631c" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 37991 } } peers { permanent_uuid: "d1c2c5c12d01441d843f06f985ca1684" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 39661 } }
I20260811 19:01:53.493518 25425 raft_consensus.cc:374] T fe0ceca7e3ec4e97a9a5ed9ea86d18c9 P d1c2c5c12d01441d843f06f985ca1684 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:01:53.493635 25425 raft_consensus.cc:729] T fe0ceca7e3ec4e97a9a5ed9ea86d18c9 P d1c2c5c12d01441d843f06f985ca1684 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1c2c5c12d01441d843f06f985ca1684, State: Initialized, Role: FOLLOWER
I20260811 19:01:53.494237 25425 consensus_queue.cc:260] T fe0ceca7e3ec4e97a9a5ed9ea86d18c9 P d1c2c5c12d01441d843f06f985ca1684 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87495242ca044b7abcd73e35b8200e67" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 46535 } } peers { permanent_uuid: "7b99824f1c6744f29b3c2787495f631c" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 37991 } } peers { permanent_uuid: "d1c2c5c12d01441d843f06f985ca1684" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 39661 } }
I20260811 19:01:53.494885 25423 raft_consensus.cc:348] T fe0ceca7e3ec4e97a9a5ed9ea86d18c9 P 7b99824f1c6744f29b3c2787495f631c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87495242ca044b7abcd73e35b8200e67" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 46535 } } peers { permanent_uuid: "7b99824f1c6744f29b3c2787495f631c" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 37991 } } peers { permanent_uuid: "d1c2c5c12d01441d843f06f985ca1684" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 39661 } }
I20260811 19:01:53.495754 25423 raft_consensus.cc:374] T fe0ceca7e3ec4e97a9a5ed9ea86d18c9 P 7b99824f1c6744f29b3c2787495f631c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:01:53.495916 25423 raft_consensus.cc:729] T fe0ceca7e3ec4e97a9a5ed9ea86d18c9 P 7b99824f1c6744f29b3c2787495f631c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b99824f1c6744f29b3c2787495f631c, State: Initialized, Role: FOLLOWER
I20260811 19:01:53.496961 25423 consensus_queue.cc:260] T fe0ceca7e3ec4e97a9a5ed9ea86d18c9 P 7b99824f1c6744f29b3c2787495f631c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87495242ca044b7abcd73e35b8200e67" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 46535 } } peers { permanent_uuid: "7b99824f1c6744f29b3c2787495f631c" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 37991 } } peers { permanent_uuid: "d1c2c5c12d01441d843f06f985ca1684" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 39661 } }
I20260811 19:01:53.502142 25391 heartbeater.cc:503] Master 127.29.112.190:45339 was elected leader, sending a full tablet report...
I20260811 19:01:53.503273 25425 ts_tablet_manager.cc:1428] T fe0ceca7e3ec4e97a9a5ed9ea86d18c9 P d1c2c5c12d01441d843f06f985ca1684: Time spent starting tablet: real 0.022s user 0.022s sys 0.000s
I20260811 19:01:53.507484 25423 ts_tablet_manager.cc:1428] T fe0ceca7e3ec4e97a9a5ed9ea86d18c9 P 7b99824f1c6744f29b3c2787495f631c: Time spent starting tablet: real 0.027s user 0.020s sys 0.008s
I20260811 19:01:53.509850 25413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:53.491499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.129:48423 (local address 127.29.112.130:37991)
0811 19:01:53.491771 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0811 19:01:53.491782 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 19:01:53.495301 (+ 3519us) server_negotiation.cc:407] Connection header received
0811 19:01:53.495792 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:01:53.495801 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:01:53.495920 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:01:53.496003 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:01:53.498503 (+ 2500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:53.500479 (+ 1976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:53.506031 (+ 5552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:53.507475 (+ 1444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:53.507610 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:53.507704 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 19:01:53.509252 (+ 1548us) server_negotiation.cc:299] Negotiation successful
0811 19:01:53.509449 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
I20260811 19:01:53.510010 25433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:53.491594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.130:37991 (local address 127.29.112.129:48423)
0811 19:01:53.495145 (+ 3551us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:53.495176 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0811 19:01:53.495460 (+ 284us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:01:53.496679 (+ 1219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:01:53.496687 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:01:53.496752 (+ 65us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:01:53.498021 (+ 1269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:01:53.498138 (+ 117us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:01:53.501009 (+ 2871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:01:53.501019 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:01:53.505789 (+ 4770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:01:53.505803 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:01:53.508082 (+ 2279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:01:53.508095 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:01:53.508868 (+ 773us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:53.508909 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 19:01:53.509210 (+ 301us) client_negotiation.cc:240] Negotiation successful
0811 19:01:53.509450 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3307,"thread_start_us":3110,"threads_started":1}
I20260811 19:01:53.510485 25179 heartbeater.cc:503] Master 127.29.112.190:45339 was elected leader, sending a full tablet report...
I20260811 19:01:53.511833 25104 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe0ceca7e3ec4e97a9a5ed9ea86d18c9" candidate_uuid: "87495242ca044b7abcd73e35b8200e67" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b99824f1c6744f29b3c2787495f631c" is_pre_election: true
I20260811 19:01:53.512552 25415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:53.495856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.129:45807 (local address 127.29.112.131:39661)
0811 19:01:53.496290 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0811 19:01:53.496307 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 19:01:53.504529 (+ 8222us) server_negotiation.cc:407] Connection header received
0811 19:01:53.504833 (+ 304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:01:53.504839 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:01:53.504911 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:01:53.504973 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:01:53.506420 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:53.507594 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:53.510369 (+ 2775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:53.511537 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:53.511640 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:53.511692 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 19:01:53.512155 (+ 463us) server_negotiation.cc:299] Negotiation successful
0811 19:01:53.512286 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116}
I20260811 19:01:53.512735 25435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:53.496517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.131:39661 (local address 127.29.112.129:45807)
0811 19:01:53.504373 (+ 7856us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:53.504396 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 19:01:53.504664 (+ 268us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:01:53.505178 (+ 514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:01:53.505183 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:01:53.505207 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:01:53.505630 (+ 423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:01:53.505649 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:01:53.507771 (+ 2122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:01:53.507776 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:01:53.509977 (+ 2201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:01:53.509999 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:01:53.511853 (+ 1854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:01:53.511857 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:01:53.511984 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:53.512006 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 19:01:53.512128 (+ 122us) client_negotiation.cc:240] Negotiation successful
0811 19:01:53.512277 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7693,"mutex_wait_us":1,"thread_start_us":11171,"threads_started":1}
I20260811 19:01:53.512789 25104 raft_consensus.cc:2454] T fe0ceca7e3ec4e97a9a5ed9ea86d18c9 P 7b99824f1c6744f29b3c2787495f631c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87495242ca044b7abcd73e35b8200e67 in term 0.
I20260811 19:01:53.513708 24798 leader_election.cc:302] T fe0ceca7e3ec4e97a9a5ed9ea86d18c9 P 87495242ca044b7abcd73e35b8200e67 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7b99824f1c6744f29b3c2787495f631c, 87495242ca044b7abcd73e35b8200e67; no voters:
I20260811 19:01:53.513844 25323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe0ceca7e3ec4e97a9a5ed9ea86d18c9" candidate_uuid: "87495242ca044b7abcd73e35b8200e67" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1c2c5c12d01441d843f06f985ca1684" is_pre_election: true
I20260811 19:01:53.514220 25432 raft_consensus.cc:2781] T fe0ceca7e3ec4e97a9a5ed9ea86d18c9 P 87495242ca044b7abcd73e35b8200e67 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 19:01:53.514406 25432 raft_consensus.cc:482] T fe0ceca7e3ec4e97a9a5ed9ea86d18c9 P 87495242ca044b7abcd73e35b8200e67 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 19:01:53.514436 25323 raft_consensus.cc:2454] T fe0ceca7e3ec4e97a9a5ed9ea86d18c9 P d1c2c5c12d01441d843f06f985ca1684 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87495242ca044b7abcd73e35b8200e67 in term 0.
I20260811 19:01:53.514634 25432 raft_consensus.cc:3037] T fe0ceca7e3ec4e97a9a5ed9ea86d18c9 P 87495242ca044b7abcd73e35b8200e67 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:01:53.518812 25432 raft_consensus.cc:504] T fe0ceca7e3ec4e97a9a5ed9ea86d18c9 P 87495242ca044b7abcd73e35b8200e67 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87495242ca044b7abcd73e35b8200e67" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 46535 } } peers { permanent_uuid: "7b99824f1c6744f29b3c2787495f631c" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 37991 } } peers { permanent_uuid: "d1c2c5c12d01441d843f06f985ca1684" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 39661 } }
I20260811 19:01:53.519773 25432 leader_election.cc:288] T fe0ceca7e3ec4e97a9a5ed9ea86d18c9 P 87495242ca044b7abcd73e35b8200e67 [CANDIDATE]: Term 1 election: Requested vote from peers 7b99824f1c6744f29b3c2787495f631c (127.29.112.130:37991), d1c2c5c12d01441d843f06f985ca1684 (127.29.112.131:39661)
I20260811 19:01:53.520484 25104 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe0ceca7e3ec4e97a9a5ed9ea86d18c9" candidate_uuid: "87495242ca044b7abcd73e35b8200e67" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b99824f1c6744f29b3c2787495f631c"
I20260811 19:01:53.520738 25104 raft_consensus.cc:3037] T fe0ceca7e3ec4e97a9a5ed9ea86d18c9 P 7b99824f1c6744f29b3c2787495f631c [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:01:53.521179 25323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe0ceca7e3ec4e97a9a5ed9ea86d18c9" candidate_uuid: "87495242ca044b7abcd73e35b8200e67" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1c2c5c12d01441d843f06f985ca1684"
I20260811 19:01:53.521569 25323 raft_consensus.cc:3037] T fe0ceca7e3ec4e97a9a5ed9ea86d18c9 P d1c2c5c12d01441d843f06f985ca1684 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:01:53.524704 25104 raft_consensus.cc:2454] T fe0ceca7e3ec4e97a9a5ed9ea86d18c9 P 7b99824f1c6744f29b3c2787495f631c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87495242ca044b7abcd73e35b8200e67 in term 1.
I20260811 19:01:53.524921 25323 raft_consensus.cc:2454] T fe0ceca7e3ec4e97a9a5ed9ea86d18c9 P d1c2c5c12d01441d843f06f985ca1684 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87495242ca044b7abcd73e35b8200e67 in term 1.
I20260811 19:01:53.525215 24798 leader_election.cc:302] T fe0ceca7e3ec4e97a9a5ed9ea86d18c9 P 87495242ca044b7abcd73e35b8200e67 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7b99824f1c6744f29b3c2787495f631c, 87495242ca044b7abcd73e35b8200e67; no voters:
I20260811 19:01:53.525607 25432 raft_consensus.cc:2781] T fe0ceca7e3ec4e97a9a5ed9ea86d18c9 P 87495242ca044b7abcd73e35b8200e67 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:01:53.525859 25432 raft_consensus.cc:686] T fe0ceca7e3ec4e97a9a5ed9ea86d18c9 P 87495242ca044b7abcd73e35b8200e67 [term 1 LEADER]: Becoming Leader. State: Replica: 87495242ca044b7abcd73e35b8200e67, State: Running, Role: LEADER
I20260811 19:01:53.526605 25432 consensus_queue.cc:237] T fe0ceca7e3ec4e97a9a5ed9ea86d18c9 P 87495242ca044b7abcd73e35b8200e67 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87495242ca044b7abcd73e35b8200e67" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 46535 } } peers { permanent_uuid: "7b99824f1c6744f29b3c2787495f631c" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 37991 } } peers { permanent_uuid: "d1c2c5c12d01441d843f06f985ca1684" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 39661 } }
I20260811 19:01:53.531656 24685 catalog_manager.cc:5495] T fe0ceca7e3ec4e97a9a5ed9ea86d18c9 P 87495242ca044b7abcd73e35b8200e67 reported cstate change: term changed from 0 to 1, leader changed from <none> to 87495242ca044b7abcd73e35b8200e67 (127.29.112.129). New cstate: current_term: 1 leader_uuid: "87495242ca044b7abcd73e35b8200e67" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87495242ca044b7abcd73e35b8200e67" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 46535 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7b99824f1c6744f29b3c2787495f631c" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 37991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1c2c5c12d01441d843f06f985ca1684" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 39661 } health_report { overall_health: UNKNOWN } } }
I20260811 19:01:53.562980 25414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:53.556410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60326 (local address 127.29.112.129:46535)
0811 19:01:53.556628 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0811 19:01:53.556632 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:01:53.556652 (+ 20us) server_negotiation.cc:407] Connection header received
0811 19:01:53.556946 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:01:53.556951 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:01:53.557274 (+ 323us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:01:53.557475 (+ 201us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:01:53.557951 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:53.558783 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:53.560377 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:53.560834 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:53.560948 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:53.561616 (+ 668us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:01:53.562196 (+ 580us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:01:53.562314 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 19:01:53.562620 (+ 306us) server_negotiation.cc:299] Negotiation successful
0811 19:01:53.562698 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
W20260811 19:01:53.570204 25392 tablet.cc:2369] T fe0ceca7e3ec4e97a9a5ed9ea86d18c9 P d1c2c5c12d01441d843f06f985ca1684: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 19:01:53.572476 25323 raft_consensus.cc:1261] T fe0ceca7e3ec4e97a9a5ed9ea86d18c9 P d1c2c5c12d01441d843f06f985ca1684 [term 1 FOLLOWER]: Refusing update from remote peer 87495242ca044b7abcd73e35b8200e67: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:01:53.572477 25104 raft_consensus.cc:1261] T fe0ceca7e3ec4e97a9a5ed9ea86d18c9 P 7b99824f1c6744f29b3c2787495f631c [term 1 FOLLOWER]: Refusing update from remote peer 87495242ca044b7abcd73e35b8200e67: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:01:53.573676 25431 consensus_queue.cc:1035] T fe0ceca7e3ec4e97a9a5ed9ea86d18c9 P 87495242ca044b7abcd73e35b8200e67 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7b99824f1c6744f29b3c2787495f631c" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 37991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:01:53.574044 25432 consensus_queue.cc:1035] T fe0ceca7e3ec4e97a9a5ed9ea86d18c9 P 87495242ca044b7abcd73e35b8200e67 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1c2c5c12d01441d843f06f985ca1684" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 39661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:01:53.587466 25439 mvcc.cc:205] Tried to move back new op lower bound from 7317401245980094464 to 7317401245809446912. Current Snapshot: MvccSnapshot[applied={T|T < 7317401245980094464}]
I20260811 19:01:53.591209 25441 mvcc.cc:205] Tried to move back new op lower bound from 7317401245980094464 to 7317401245809446912. Current Snapshot: MvccSnapshot[applied={T|T < 7317401245980094464}]
I20260811 19:01:53.605922 24683 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33472:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:01:53.606343 24683 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33472:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:01:53.611496 24683 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 394f0e9f175e4f24b7e1081aaeb6fb3b: Sending DeleteTablet for 3 replicas of tablet fe0ceca7e3ec4e97a9a5ed9ea86d18c9
I20260811 19:01:53.613014 24860 tablet_service.cc:1514] Processing DeleteTablet for tablet fe0ceca7e3ec4e97a9a5ed9ea86d18c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:01:53 UTC) from {username='kudu', principal='kudu/127.29.112.190@KRBTEST.COM'} at 127.0.0.1:60310
I20260811 19:01:53.613071 25076 tablet_service.cc:1514] Processing DeleteTablet for tablet fe0ceca7e3ec4e97a9a5ed9ea86d18c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:01:53 UTC) from {username='kudu', principal='kudu/127.29.112.190@KRBTEST.COM'} at 127.0.0.1:60790
I20260811 19:01:53.613088 25290 tablet_service.cc:1514] Processing DeleteTablet for tablet fe0ceca7e3ec4e97a9a5ed9ea86d18c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:01:53 UTC) from {username='kudu', principal='kudu/127.29.112.190@KRBTEST.COM'} at 127.0.0.1:36654
I20260811 19:01:53.613214 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 24743
I20260811 19:01:53.626106 25457 tablet_replica.cc:330] T fe0ceca7e3ec4e97a9a5ed9ea86d18c9 P 7b99824f1c6744f29b3c2787495f631c: stopping tablet replica
I20260811 19:01:53.626555 25459 tablet_replica.cc:330] T fe0ceca7e3ec4e97a9a5ed9ea86d18c9 P 87495242ca044b7abcd73e35b8200e67: stopping tablet replica
I20260811 19:01:53.626916 25457 raft_consensus.cc:2229] T fe0ceca7e3ec4e97a9a5ed9ea86d18c9 P 7b99824f1c6744f29b3c2787495f631c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:01:53.627449 25459 raft_consensus.cc:2229] T fe0ceca7e3ec4e97a9a5ed9ea86d18c9 P 87495242ca044b7abcd73e35b8200e67 [term 1 LEADER]: Raft consensus shutting down.
I20260811 19:01:53.627789 25457 raft_consensus.cc:2258] T fe0ceca7e3ec4e97a9a5ed9ea86d18c9 P 7b99824f1c6744f29b3c2787495f631c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:01:53.628255 25459 raft_consensus.cc:2258] T fe0ceca7e3ec4e97a9a5ed9ea86d18c9 P 87495242ca044b7abcd73e35b8200e67 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:01:53.630934 25459 ts_tablet_manager.cc:1905] T fe0ceca7e3ec4e97a9a5ed9ea86d18c9 P 87495242ca044b7abcd73e35b8200e67: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:01:53.632164 25457 ts_tablet_manager.cc:1905] T fe0ceca7e3ec4e97a9a5ed9ea86d18c9 P 7b99824f1c6744f29b3c2787495f631c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:01:53.637972 25414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:53.615168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60336 (local address 127.29.112.129:46535)
0811 19:01:53.615410 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0811 19:01:53.615418 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:01:53.615441 (+ 23us) server_negotiation.cc:407] Connection header received
0811 19:01:53.615515 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:01:53.615523 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:01:53.615645 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:01:53.615767 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:01:53.618282 (+ 2515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:53.620126 (+ 1844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:53.622398 (+ 2272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:53.623175 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:53.623322 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:53.628279 (+ 4957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:01:53.628295 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:01:53.628315 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:01:53.628382 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:01:53.633106 (+ 4724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:01:53.634390 (+ 1284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:01:53.634400 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:01:53.634404 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:01:53.634612 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:01:53.635107 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:01:53.635112 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:01:53.635115 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:01:53.635682 (+ 567us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:01:53.635996 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0811 19:01:53.636931 (+ 935us) server_negotiation.cc:299] Negotiation successful
0811 19:01:53.637197 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 19:01:53.640564 24954 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:01:53.641039 25460 tablet_replica.cc:330] T fe0ceca7e3ec4e97a9a5ed9ea86d18c9 P d1c2c5c12d01441d843f06f985ca1684: stopping tablet replica
I20260811 19:01:53.641345 25457 ts_tablet_manager.cc:1918] T fe0ceca7e3ec4e97a9a5ed9ea86d18c9 P 7b99824f1c6744f29b3c2787495f631c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:01:53.641546 25457 log.cc:1196] T fe0ceca7e3ec4e97a9a5ed9ea86d18c9 P 7b99824f1c6744f29b3c2787495f631c: Deleting WAL directory at /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-1/wal/wals/fe0ceca7e3ec4e97a9a5ed9ea86d18c9
I20260811 19:01:53.641592 25460 raft_consensus.cc:2229] T fe0ceca7e3ec4e97a9a5ed9ea86d18c9 P d1c2c5c12d01441d843f06f985ca1684 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:01:53.642241 25457 ts_tablet_manager.cc:1939] T fe0ceca7e3ec4e97a9a5ed9ea86d18c9 P 7b99824f1c6744f29b3c2787495f631c: Deleting consensus metadata
I20260811 19:01:53.642283 25460 raft_consensus.cc:2258] T fe0ceca7e3ec4e97a9a5ed9ea86d18c9 P d1c2c5c12d01441d843f06f985ca1684 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:01:53.644826 24665 catalog_manager.cc:4841] TS 7b99824f1c6744f29b3c2787495f631c (127.29.112.130:37991): tablet fe0ceca7e3ec4e97a9a5ed9ea86d18c9 (table test-table [id=c8afcc1e9ddf4d3f99a2fcd0263d0595]) successfully deleted
I20260811 19:01:53.645678 25460 ts_tablet_manager.cc:1905] T fe0ceca7e3ec4e97a9a5ed9ea86d18c9 P d1c2c5c12d01441d843f06f985ca1684: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:01:53.652899 25460 ts_tablet_manager.cc:1918] T fe0ceca7e3ec4e97a9a5ed9ea86d18c9 P d1c2c5c12d01441d843f06f985ca1684: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:01:53.653053 25460 log.cc:1196] T fe0ceca7e3ec4e97a9a5ed9ea86d18c9 P d1c2c5c12d01441d843f06f985ca1684: Deleting WAL directory at /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-2/wal/wals/fe0ceca7e3ec4e97a9a5ed9ea86d18c9
I20260811 19:01:53.653528 25460 ts_tablet_manager.cc:1939] T fe0ceca7e3ec4e97a9a5ed9ea86d18c9 P d1c2c5c12d01441d843f06f985ca1684: Deleting consensus metadata
I20260811 19:01:53.655545 24665 catalog_manager.cc:4841] TS d1c2c5c12d01441d843f06f985ca1684 (127.29.112.131:39661): tablet fe0ceca7e3ec4e97a9a5ed9ea86d18c9 (table test-table [id=c8afcc1e9ddf4d3f99a2fcd0263d0595]) successfully deleted
I20260811 19:01:53.752671 25459 ts_tablet_manager.cc:1918] T fe0ceca7e3ec4e97a9a5ed9ea86d18c9 P 87495242ca044b7abcd73e35b8200e67: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:01:53.753001 25459 log.cc:1196] T fe0ceca7e3ec4e97a9a5ed9ea86d18c9 P 87495242ca044b7abcd73e35b8200e67: Deleting WAL directory at /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-0/wal/wals/fe0ceca7e3ec4e97a9a5ed9ea86d18c9
I20260811 19:01:53.753427 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 24743
W20260811 19:01:53.789811 24664 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.112.129:46535 (error 108)
I20260811 19:01:53.790045 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 24975
I20260811 19:01:53.792965 25416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:53.791700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:46535 (local address 127.0.0.1:60348)
0811 19:01:53.792396 (+ 696us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:53.792740 (+ 344us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.129:46535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":302}
W20260811 19:01:53.793495 24664 catalog_manager.cc:4584] TS 87495242ca044b7abcd73e35b8200e67 (127.29.112.129:46535): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet fe0ceca7e3ec4e97a9a5ed9ea86d18c9: Network error: Client connection negotiation failed: client connection to 127.29.112.129:46535: connect: Connection refused (error 111)
I20260811 19:01:53.809921 25413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:53.790989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60802 (local address 127.29.112.130:37991)
0811 19:01:53.791228 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0811 19:01:53.791236 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:01:53.791445 (+ 209us) server_negotiation.cc:407] Connection header received
0811 19:01:53.791656 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:01:53.791660 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:01:53.791728 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:01:53.791875 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:01:53.794303 (+ 2428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:53.795132 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:53.796417 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:53.796802 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:53.796945 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:53.800257 (+ 3312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:01:53.800271 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:01:53.800288 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:01:53.800332 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:01:53.805659 (+ 5327us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:01:53.807124 (+ 1465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:01:53.807133 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:01:53.807136 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:01:53.807323 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:01:53.807765 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:01:53.807771 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:01:53.807775 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:01:53.808314 (+ 539us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:01:53.808670 (+ 356us) server_negotiation.cc:1035] Waiting for connection context
0811 19:01:53.809022 (+ 352us) server_negotiation.cc:299] Negotiation successful
0811 19:01:53.809296 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260811 19:01:53.811075 25171 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:01:53.847702 25416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:53.847215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:46535 (local address 127.0.0.1:60358)
0811 19:01:53.847438 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:53.847553 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.129:46535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260811 19:01:53.849002 25416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:53.848549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:46535 (local address 127.0.0.1:60370)
0811 19:01:53.848749 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:53.848837 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.129:46535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260811 19:01:53.910058 25416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:53.909177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:46535 (local address 127.0.0.1:60372)
0811 19:01:53.909611 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:53.909797 (+ 186us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.129:46535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":245}
I20260811 19:01:53.911655 25416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:53.911159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:46535 (local address 127.0.0.1:60382)
0811 19:01:53.911370 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:53.911483 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.129:46535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260811 19:01:53.915447 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 24975
I20260811 19:01:53.948657 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 25184
I20260811 19:01:53.966948 25415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:53.949569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36670 (local address 127.29.112.131:39661)
0811 19:01:53.949760 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 19:01:53.949764 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:01:53.949785 (+ 21us) server_negotiation.cc:407] Connection header received
0811 19:01:53.950063 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:01:53.950068 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:01:53.950138 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:01:53.950231 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:01:53.952546 (+ 2315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:53.953695 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:53.955600 (+ 1905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:01:53.956102 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:01:53.956261 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:01:53.960059 (+ 3798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:01:53.960074 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:01:53.960091 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:01:53.960128 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:01:53.963169 (+ 3041us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:01:53.963970 (+ 801us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:01:53.963978 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:01:53.963986 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:01:53.964159 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:01:53.964708 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:01:53.964712 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:01:53.964715 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:01:53.965142 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:01:53.965396 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0811 19:01:53.966161 (+ 765us) server_negotiation.cc:299] Negotiation successful
0811 19:01:53.966651 (+ 490us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 19:01:53.968200 25386 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:01:53.992780 25416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:53.992044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:46535 (local address 127.0.0.1:60394)
0811 19:01:53.992506 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:53.992610 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.129:46535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":316}
I20260811 19:01:53.993983 25416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:53.993571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:46535 (local address 127.0.0.1:60402)
0811 19:01:53.993793 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:53.993873 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.129:46535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
I20260811 19:01:54.073058 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 25184
I20260811 19:01:54.103372 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 24638
I20260811 19:01:54.104667 24734 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:01:54.227057 25416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:01:54.226560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:46535 (local address 127.0.0.1:60406)
0811 19:01:54.226783 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0811 19:01:54.226890 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.129:46535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260811 19:01:54.227725 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 24638
I20260811 19:02:14.265019 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.190:45339
--webserver_interface=127.29.112.190
--webserver_port=43159
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab
--principal=kudu/127.29.112.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:36075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.190:45339
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:02:14.667025 25467 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:02:14.667680 25467 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:02:14.667814 25467 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:02:14.678319 25467 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 19:02:14.678519 25467 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 19:02:14.678678 25467 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 19:02:14.678755 25467 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:02:14.678833 25467 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:02:14.678926 25467 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:02:14.678987 25467 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:02:14.679046 25467 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:02:14.696096 25467 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:36075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.29.112.190:45339
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab
--principal=kudu/127.29.112.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.190:45339
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.112.190
--webserver_port=43159
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:02:14.699499 25467 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:02:14.701651 25467 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:02:14.722052 25476 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:02:14.722311 25481 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:02:14.722081 25473 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:02:14.727859 25467 server_base.cc:1034] running on GCE node
I20260811 19:02:14.729837 25467 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:02:14.731837 25467 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:02:14.733323 25467 hybrid_clock.cc:648] HybridClock initialized: now 1786474934733084 us; error 67 us; skew 500 ppm
I20260811 19:02:14.733951 25467 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:02:14 dist-test-slave-h6n0 krb5kdc[23787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474934, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:02:14.738358 25467 init.cc:377] Logged in from keytab as kudu/127.29.112.190@KRBTEST.COM (short username kudu)
I20260811 19:02:14.740379 25467 webserver.cc:460] Webserver started at http://127.29.112.190:43159/ using document root <none> and password file <none>
I20260811 19:02:14.741282 25467 fs_manager.cc:359] Metadata directory not provided
I20260811 19:02:14.741454 25467 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:02:14.748615 25467 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.004s
I20260811 19:02:14.752533 25494 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:14.754655 25467 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.001s
I20260811 19:02:14.754885 25467 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/master-0/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/master-0/wal
uuid: "394f0e9f175e4f24b7e1081aaeb6fb3b"
format_stamp: "Formatted at 2026-08-11 19:01:27 on dist-test-slave-h6n0"
server_key: "54b1f8faf9111f2625fb7e8489539574"
server_key_iv: "69b659430a273751b5130371a7442b26"
server_key_version: "encryptionkey@0"
I20260811 19:02:14.755968 25467 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:02:14.774164 25467 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:02:14.780892 25467 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:02:14.781270 25467 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:02:14.811328 25467 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.190:45339
I20260811 19:02:14.811328 25579 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.190:45339 every 8 connection(s)
I20260811 19:02:14.813195 25467 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
I20260811 19:02:14.814186 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 25467
I20260811 19:02:14.815897 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.129:46535
--local_ip_for_outbound_sockets=127.29.112.129
--tserver_master_addrs=127.29.112.190:45339
--webserver_port=45245
--webserver_interface=127.29.112.129
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.129.keytab
--principal=kudu/127.29.112.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:36075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 19:02:14.822773 25580 sys_catalog.cc:263] Verifying existing consensus state
I20260811 19:02:14.826712 25580 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 394f0e9f175e4f24b7e1081aaeb6fb3b: Bootstrap starting.
I20260811 19:02:14.857767 25580 log.cc:824] T 00000000000000000000000000000000 P 394f0e9f175e4f24b7e1081aaeb6fb3b: Log is configured to *not* fsync() on all Append() calls
I20260811 19:02:14.881378 25580 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 394f0e9f175e4f24b7e1081aaeb6fb3b: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 19:02:14.882339 25580 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 394f0e9f175e4f24b7e1081aaeb6fb3b: Bootstrap complete.
I20260811 19:02:14.893761 25580 raft_consensus.cc:348] T 00000000000000000000000000000000 P 394f0e9f175e4f24b7e1081aaeb6fb3b [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "394f0e9f175e4f24b7e1081aaeb6fb3b" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 45339 } }
I20260811 19:02:14.895499 25580 raft_consensus.cc:729] T 00000000000000000000000000000000 P 394f0e9f175e4f24b7e1081aaeb6fb3b [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 394f0e9f175e4f24b7e1081aaeb6fb3b, State: Initialized, Role: FOLLOWER
I20260811 19:02:14.896937 25580 consensus_queue.cc:260] T 00000000000000000000000000000000 P 394f0e9f175e4f24b7e1081aaeb6fb3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "394f0e9f175e4f24b7e1081aaeb6fb3b" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 45339 } }
I20260811 19:02:14.897295 25580 raft_consensus.cc:388] T 00000000000000000000000000000000 P 394f0e9f175e4f24b7e1081aaeb6fb3b [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:02:14.897492 25580 raft_consensus.cc:482] T 00000000000000000000000000000000 P 394f0e9f175e4f24b7e1081aaeb6fb3b [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:02:14.897718 25580 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 394f0e9f175e4f24b7e1081aaeb6fb3b [term 2 FOLLOWER]: Advancing to term 3
I20260811 19:02:14.902873 25580 raft_consensus.cc:504] T 00000000000000000000000000000000 P 394f0e9f175e4f24b7e1081aaeb6fb3b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "394f0e9f175e4f24b7e1081aaeb6fb3b" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 45339 } }
I20260811 19:02:14.903534 25580 leader_election.cc:302] T 00000000000000000000000000000000 P 394f0e9f175e4f24b7e1081aaeb6fb3b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 394f0e9f175e4f24b7e1081aaeb6fb3b; no voters:
I20260811 19:02:14.904335 25580 leader_election.cc:288] T 00000000000000000000000000000000 P 394f0e9f175e4f24b7e1081aaeb6fb3b [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 19:02:14.904510 25589 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 394f0e9f175e4f24b7e1081aaeb6fb3b [term 3 FOLLOWER]: Leader election won for term 3
I20260811 19:02:14.906311 25589 raft_consensus.cc:686] T 00000000000000000000000000000000 P 394f0e9f175e4f24b7e1081aaeb6fb3b [term 3 LEADER]: Becoming Leader. State: Replica: 394f0e9f175e4f24b7e1081aaeb6fb3b, State: Running, Role: LEADER
I20260811 19:02:14.907404 25580 sys_catalog.cc:564] T 00000000000000000000000000000000 P 394f0e9f175e4f24b7e1081aaeb6fb3b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:02:14.907259 25589 consensus_queue.cc:237] T 00000000000000000000000000000000 P 394f0e9f175e4f24b7e1081aaeb6fb3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "394f0e9f175e4f24b7e1081aaeb6fb3b" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 45339 } }
I20260811 19:02:14.910887 25591 sys_catalog.cc:455] T 00000000000000000000000000000000 P 394f0e9f175e4f24b7e1081aaeb6fb3b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "394f0e9f175e4f24b7e1081aaeb6fb3b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "394f0e9f175e4f24b7e1081aaeb6fb3b" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 45339 } } }
I20260811 19:02:14.912700 25591 sys_catalog.cc:458] T 00000000000000000000000000000000 P 394f0e9f175e4f24b7e1081aaeb6fb3b [sys.catalog]: This master's current role is: LEADER
I20260811 19:02:14.911963 25592 sys_catalog.cc:455] T 00000000000000000000000000000000 P 394f0e9f175e4f24b7e1081aaeb6fb3b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 394f0e9f175e4f24b7e1081aaeb6fb3b. Latest consensus state: current_term: 3 leader_uuid: "394f0e9f175e4f24b7e1081aaeb6fb3b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "394f0e9f175e4f24b7e1081aaeb6fb3b" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 45339 } } }
I20260811 19:02:14.913579 25592 sys_catalog.cc:458] T 00000000000000000000000000000000 P 394f0e9f175e4f24b7e1081aaeb6fb3b [sys.catalog]: This master's current role is: LEADER
I20260811 19:02:14.919054 25596 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:02:14.933434 25596 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:02:14.936298 25596 catalog_manager.cc:1259] Loaded cluster ID: 79b877f1f5ac49f4907b1d7cb27744fa
I20260811 19:02:14.936463 25596 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:02:14.944068 25596 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:02:14.947000 25596 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 394f0e9f175e4f24b7e1081aaeb6fb3b: Loaded TSK: 0
I20260811 19:02:14.947252 25596 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 394f0e9f175e4f24b7e1081aaeb6fb3b: Loaded TSK: 1
F20260811 19:02:14.949853 25596 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f0e017609ad google::LogMessage::Fail() at ??:0
I20260811 19:02:14.958312 25616 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
@ 0x7f0e0176388a google::LogMessage::SendToLog() at ??:0
@ 0x7f0e017604de google::LogMessage::Flush() at ??:0
@ 0x7f0e01761a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f0e054c47d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f0e054b373e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f0e1718f1c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f0e1718e783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f0e1721a2a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7f0e1721a0c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7f0e172645e9 std::function<>::operator()() at ??:0
@ 0x7f0e17192cdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f0e17190c36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f0e17218610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f0e17218439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f0e043ac345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f0e043af91c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f0e043af799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f0e0438aeaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f0e076646db start_thread at ??:0
@ 0x7f0e005fa61f clone at ??:0
W20260811 19:02:15.315207 25584 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:02:15.315613 25584 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:02:15.315809 25584 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:02:15.324870 25584 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:02:15.325048 25584 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:02:15.325241 25584 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.129
I20260811 19:02:15.337360 25584 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:36075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.129.keytab
--principal=kudu/127.29.112.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.129:46535
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.112.129
--webserver_port=45245
--tserver_master_addrs=127.29.112.190:45339
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.129
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:02:15.338838 25584 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:02:15.340519 25584 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:02:15.356155 25655 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:02:15.356526 25656 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:02:15.356150 25658 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:02:15.357743 25584 server_base.cc:1034] running on GCE node
I20260811 19:02:15.358507 25584 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:02:15.359637 25584 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:02:15.360966 25584 hybrid_clock.cc:648] HybridClock initialized: now 1786474935360870 us; error 90 us; skew 500 ppm
I20260811 19:02:15.361344 25584 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:02:15 dist-test-slave-h6n0 krb5kdc[23787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474935, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:02:15.365319 25584 init.cc:377] Logged in from keytab as kudu/127.29.112.129@KRBTEST.COM (short username kudu)
I20260811 19:02:15.367331 25584 webserver.cc:460] Webserver started at http://127.29.112.129:45245/ using document root <none> and password file <none>
I20260811 19:02:15.368180 25584 fs_manager.cc:359] Metadata directory not provided
I20260811 19:02:15.368355 25584 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:02:15.375370 25584 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 19:02:15.379146 25670 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:15.381031 25584 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260811 19:02:15.381255 25584 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-0/wal
uuid: "87495242ca044b7abcd73e35b8200e67"
format_stamp: "Formatted at 2026-08-11 19:01:28 on dist-test-slave-h6n0"
server_key: "e3e7ca1842fbc341318292979be8d2b0"
server_key_iv: "778c5330775e38609965abf788fc5b59"
server_key_version: "encryptionkey@0"
I20260811 19:02:15.382447 25584 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:02:15.413364 25584 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:02:15.418445 25584 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:02:15.418830 25584 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:02:15.419974 25584 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:02:15.422109 25685 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
W20260811 19:02:15.427908 25685 ts_tablet_manager.cc:1852] T fe0ceca7e3ec4e97a9a5ed9ea86d18c9 P 87495242ca044b7abcd73e35b8200e67: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260811 19:02:15.428998 25685 ts_tablet_manager.cc:1905] T fe0ceca7e3ec4e97a9a5ed9ea86d18c9 P 87495242ca044b7abcd73e35b8200e67: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:02:15.436877 25685 ts_tablet_manager.cc:1918] T fe0ceca7e3ec4e97a9a5ed9ea86d18c9 P 87495242ca044b7abcd73e35b8200e67: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:02:15.437173 25685 ts_tablet_manager.cc:1939] T fe0ceca7e3ec4e97a9a5ed9ea86d18c9 P 87495242ca044b7abcd73e35b8200e67: Deleting consensus metadata
I20260811 19:02:15.437608 25584 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260811 19:02:15.437832 25584 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.016s user 0.001s sys 0.000s
I20260811 19:02:15.438424 25584 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:02:15.438882 25584 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.001s
I20260811 19:02:15.483485 25836 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.129:46535 every 8 connection(s)
I20260811 19:02:15.483494 25584 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.129:46535
I20260811 19:02:15.485306 25584 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-0/data/info.pb
I20260811 19:02:15.491650 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 25584
I20260811 19:02:15.491909 25842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:15.489640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:45339 (local address 127.29.112.129:32783)
0811 19:02:15.490498 (+ 858us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:15.490903 (+ 405us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":609,"thread_start_us":390,"threads_started":1}
I20260811 19:02:15.492621 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.130:37991
--local_ip_for_outbound_sockets=127.29.112.130
--tserver_master_addrs=127.29.112.190:45339
--webserver_port=33725
--webserver_interface=127.29.112.130
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.130.keytab
--principal=kudu/127.29.112.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:36075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:02:15.492712 25837 heartbeater.cc:644] Failed to heartbeat to 127.29.112.190:45339 (0 consecutive failures): Network error: Failed to ping master at 127.29.112.190:45339: Client connection negotiation failed: client connection to 127.29.112.190:45339: connect: Connection refused (error 111)
I20260811 19:02:15.493707 25842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:15.493346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:45339 (local address 127.29.112.129:57293)
0811 19:02:15.493532 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:15.493605 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260811 19:02:15.494680 25842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:15.494241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:45339 (local address 127.29.112.129:40713)
0811 19:02:15.494388 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:15.494483 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260811 19:02:15.494882 25837 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 19:02:15.925501 25843 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:02:15.925836 25843 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:02:15.925947 25843 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:02:15.933743 25843 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:02:15.934046 25843 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:02:15.934231 25843 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.130
I20260811 19:02:15.945637 25843 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:36075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.130.keytab
--principal=kudu/127.29.112.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.130:37991
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.112.130
--webserver_port=33725
--tserver_master_addrs=127.29.112.190:45339
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.130
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:02:15.947093 25843 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:02:15.948496 25843 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:02:15.964105 25850 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:02:15.964351 25851 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:02:15.964367 25853 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:02:15.966355 25843 server_base.cc:1034] running on GCE node
I20260811 19:02:15.967437 25843 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:02:15.969103 25843 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:02:15.970513 25843 hybrid_clock.cc:648] HybridClock initialized: now 1786474935970433 us; error 69 us; skew 500 ppm
I20260811 19:02:15.971369 25843 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:02:15 dist-test-slave-h6n0 krb5kdc[23787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474935, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:02:15.975397 25843 init.cc:377] Logged in from keytab as kudu/127.29.112.130@KRBTEST.COM (short username kudu)
I20260811 19:02:15.977255 25843 webserver.cc:460] Webserver started at http://127.29.112.130:33725/ using document root <none> and password file <none>
I20260811 19:02:15.978039 25843 fs_manager.cc:359] Metadata directory not provided
I20260811 19:02:15.978344 25843 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:02:15.986843 25843 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.003s sys 0.004s
I20260811 19:02:15.991380 25865 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:15.993218 25843 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 19:02:15.993458 25843 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-1/wal
uuid: "7b99824f1c6744f29b3c2787495f631c"
format_stamp: "Formatted at 2026-08-11 19:01:28 on dist-test-slave-h6n0"
server_key: "5d7517d24d4f31e365c04e607ea07445"
server_key_iv: "205d31e6af17ae27cd13e7d9574e79cc"
server_key_version: "encryptionkey@0"
I20260811 19:02:15.994637 25843 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:02:16.069041 25843 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:02:16.076467 25843 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:02:16.076892 25843 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:02:16.078634 25843 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:02:16.080857 25843 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:02:16.081146 25843 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:16.081346 25843 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:02:16.081496 25843 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:16.136135 25843 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.130:37991
I20260811 19:02:16.136438 26056 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.130:37991 every 8 connection(s)
I20260811 19:02:16.139739 25843 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-1/data/info.pb
I20260811 19:02:16.147854 26059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:16.144626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:45339 (local address 127.29.112.130:33991)
0811 19:02:16.146112 (+ 1486us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:16.146810 (+ 698us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1034,"thread_start_us":505,"threads_started":1}
W20260811 19:02:16.148830 26057 heartbeater.cc:644] Failed to heartbeat to 127.29.112.190:45339 (0 consecutive failures): Network error: Failed to ping master at 127.29.112.190:45339: Client connection negotiation failed: client connection to 127.29.112.190:45339: connect: Connection refused (error 111)
I20260811 19:02:16.149972 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 25843
I20260811 19:02:16.150455 26059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:16.149872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:45339 (local address 127.29.112.130:55691)
0811 19:02:16.150214 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:16.150318 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148}
I20260811 19:02:16.151726 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.131:39661
--local_ip_for_outbound_sockets=127.29.112.131
--tserver_master_addrs=127.29.112.190:45339
--webserver_port=41429
--webserver_interface=127.29.112.131
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.131.keytab
--principal=kudu/127.29.112.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:36075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 19:02:16.151948 26059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:16.151206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:45339 (local address 127.29.112.130:50405)
0811 19:02:16.151535 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:16.151705 (+ 170us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154}
W20260811 19:02:16.152645 26057 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 19:02:16.496820 26064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:16.495801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:45339 (local address 127.29.112.129:32841)
0811 19:02:16.496439 (+ 638us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:16.496567 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":352,"threads_started":1}
W20260811 19:02:16.587785 26060 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:02:16.588379 26060 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:02:16.588618 26060 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:02:16.599355 26060 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:02:16.599550 26060 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:02:16.599763 26060 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.131
I20260811 19:02:16.612944 26060 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:36075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.131.keytab
--principal=kudu/127.29.112.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.131:39661
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.112.131
--webserver_port=41429
--tserver_master_addrs=127.29.112.190:45339
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.131
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:02:16.614598 26060 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:02:16.616147 26060 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:02:16.633821 26077 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:02:16.633772 26086 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:02:16.634042 26079 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:02:16.636197 26060 server_base.cc:1034] running on GCE node
I20260811 19:02:16.637405 26060 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:02:16.638921 26060 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:02:16.640244 26060 hybrid_clock.cc:648] HybridClock initialized: now 1786474936640188 us; error 59 us; skew 500 ppm
I20260811 19:02:16.640702 26060 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:02:16 dist-test-slave-h6n0 krb5kdc[23787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474936, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:02:16.644896 26060 init.cc:377] Logged in from keytab as kudu/127.29.112.131@KRBTEST.COM (short username kudu)
I20260811 19:02:16.647024 26060 webserver.cc:460] Webserver started at http://127.29.112.131:41429/ using document root <none> and password file <none>
I20260811 19:02:16.647871 26060 fs_manager.cc:359] Metadata directory not provided
I20260811 19:02:16.648044 26060 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:02:16.655799 26060 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 19:02:16.659627 26095 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:16.661671 26060 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 19:02:16.661955 26060 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-2/wal
uuid: "d1c2c5c12d01441d843f06f985ca1684"
format_stamp: "Formatted at 2026-08-11 19:01:29 on dist-test-slave-h6n0"
server_key: "391405d26b86085610c85f500acb08b8"
server_key_iv: "35bf95b36cd7cb7ba01aefd9acfcf118"
server_key_version: "encryptionkey@0"
I20260811 19:02:16.663237 26060 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:02:16.688638 26060 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:02:16.694589 26060 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:02:16.694938 26060 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:02:16.696161 26060 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:02:16.697878 26060 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:02:16.698022 26060 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:16.698195 26060 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:02:16.698307 26060 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:16.746093 26060 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.131:39661
I20260811 19:02:16.746109 26274 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.131:39661 every 8 connection(s)
I20260811 19:02:16.748781 26060 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474615205622-30146-0/minicluster-data/ts-2/data/info.pb
I20260811 19:02:16.756320 26281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:16.753766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:45339 (local address 127.29.112.131:49993)
0811 19:02:16.754791 (+ 1025us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:16.755382 (+ 591us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":608,"thread_start_us":413,"threads_started":1}
W20260811 19:02:16.757192 26276 heartbeater.cc:644] Failed to heartbeat to 127.29.112.190:45339 (0 consecutive failures): Network error: Failed to ping master at 127.29.112.190:45339: Client connection negotiation failed: client connection to 127.29.112.190:45339: connect: Connection refused (error 111)
I20260811 19:02:16.758649 26281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:16.757872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:45339 (local address 127.29.112.131:38015)
0811 19:02:16.758216 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:16.758362 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133}
I20260811 19:02:16.759588 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 26060
I20260811 19:02:16.760301 26281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:16.759813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:45339 (local address 127.29.112.131:39201)
0811 19:02:16.760035 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:16.760137 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
W20260811 19:02:16.760761 26276 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 19:02:17.154203 26283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:17.153294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:45339 (local address 127.29.112.130:44603)
0811 19:02:17.153817 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:17.153942 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":318,"threads_started":1}
I20260811 19:02:17.499508 26285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:17.498087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:45339 (local address 127.29.112.129:60901)
0811 19:02:17.499005 (+ 918us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:17.499245 (+ 240us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":663,"thread_start_us":522,"threads_started":1}
I20260811 19:02:17.762358 26286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:17.761370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:45339 (local address 127.29.112.131:59953)
0811 19:02:17.761918 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:17.762034 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":264,"threads_started":1}
I20260811 19:02:18.156383 26287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:18.155539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:45339 (local address 127.29.112.130:37799)
0811 19:02:18.156091 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:18.156186 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":339,"threads_started":1}
I20260811 19:02:18.501721 26290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:18.500838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:45339 (local address 127.29.112.129:39987)
0811 19:02:18.501385 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:18.501503 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":290,"threads_started":1}
I20260811 19:02:18.764282 26292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:18.763526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:45339 (local address 127.29.112.131:48957)
0811 19:02:18.764004 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:18.764098 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":237,"threads_started":1}
I20260811 19:02:19.158277 26293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:19.157458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:45339 (local address 127.29.112.130:42629)
0811 19:02:19.157993 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:19.158086 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":262,"threads_started":1}
I20260811 19:02:19.504273 26294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:19.503138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:45339 (local address 127.29.112.129:60933)
0811 19:02:19.503681 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:19.504060 (+ 379us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":287,"threads_started":1}
I20260811 19:02:19.766738 26296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:19.765845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:45339 (local address 127.29.112.131:47989)
0811 19:02:19.766405 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:19.766487 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":301,"threads_started":1}
I20260811 19:02:20.160871 26297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:20.159769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:45339 (local address 127.29.112.130:53125)
0811 19:02:20.160556 (+ 787us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:20.160665 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":652,"thread_start_us":533,"threads_started":1}
I20260811 19:02:20.506775 26298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:20.505687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:45339 (local address 127.29.112.129:45205)
0811 19:02:20.506262 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:20.506358 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":296,"threads_started":1}
I20260811 19:02:20.768697 26299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:20.767881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:45339 (local address 127.29.112.131:57917)
0811 19:02:20.768401 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:20.768489 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":281,"threads_started":1}
I20260811 19:02:21.162880 26300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:21.161970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:45339 (local address 127.29.112.130:52611)
0811 19:02:21.162522 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:21.162682 (+ 160us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":288,"threads_started":1}
I20260811 19:02:21.509092 26301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:21.508094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:45339 (local address 127.29.112.129:57397)
0811 19:02:21.508750 (+ 656us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:21.508861 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":352,"threads_started":1}
I20260811 19:02:21.770923 26302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:21.770058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:45339 (local address 127.29.112.131:33901)
0811 19:02:21.770665 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:21.770746 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":310,"threads_started":1}
I20260811 19:02:22.164947 26303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:22.164065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:45339 (local address 127.29.112.130:37933)
0811 19:02:22.164627 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:22.164738 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":283,"threads_started":1}
I20260811 19:02:22.511516 26304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:22.510388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:45339 (local address 127.29.112.129:54405)
0811 19:02:22.511158 (+ 770us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:22.511276 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":652,"thread_start_us":408,"threads_started":1}
I20260811 19:02:22.773350 26306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:22.771976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:45339 (local address 127.29.112.131:50669)
0811 19:02:22.773068 (+ 1092us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:22.773171 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":983,"thread_start_us":829,"threads_started":1}
I20260811 19:02:23.167358 26307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:23.166340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:45339 (local address 127.29.112.130:36659)
0811 19:02:23.166978 (+ 638us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:23.167076 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":319,"threads_started":1}
I20260811 19:02:23.514222 26308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:23.512960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:45339 (local address 127.29.112.129:54655)
0811 19:02:23.513728 (+ 768us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:23.513896 (+ 168us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":613,"thread_start_us":429,"threads_started":1}
I20260811 19:02:23.775666 26311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:23.774715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:45339 (local address 127.29.112.131:56659)
0811 19:02:23.775340 (+ 625us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:23.775434 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":304,"threads_started":1}
I20260811 19:02:24.169428 26312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:24.168597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:45339 (local address 127.29.112.130:47649)
0811 19:02:24.169145 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:24.169241 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":313,"threads_started":1}
I20260811 19:02:24.516645 26314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:24.515814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:45339 (local address 127.29.112.129:53215)
0811 19:02:24.516352 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:24.516462 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":269,"threads_started":1}
I20260811 19:02:24.777546 26316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:24.776670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:45339 (local address 127.29.112.131:53839)
0811 19:02:24.777193 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:24.777307 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":238,"threads_started":1}
I20260811 19:02:25.171538 26319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:25.170626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:45339 (local address 127.29.112.130:39405)
0811 19:02:25.171255 (+ 629us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:25.171344 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":347,"threads_started":1}
I20260811 19:02:25.518483 26323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:25.517759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:45339 (local address 127.29.112.129:57757)
0811 19:02:25.518211 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:25.518297 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":236,"threads_started":1}
I20260811 19:02:25.779711 26324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:25.778816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:45339 (local address 127.29.112.131:41865)
0811 19:02:25.779444 (+ 628us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:25.779529 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":397,"threads_started":1}
I20260811 19:02:26.173460 26327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:26.172548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:45339 (local address 127.29.112.130:38903)
0811 19:02:26.173145 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:26.173250 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":335,"threads_started":1}
I20260811 19:02:26.520843 26328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:26.519641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:45339 (local address 127.29.112.129:35405)
0811 19:02:26.520452 (+ 811us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:26.520594 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":650,"thread_start_us":369,"threads_started":1}
I20260811 19:02:26.781956 26330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:26.780987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:45339 (local address 127.29.112.131:51709)
0811 19:02:26.781672 (+ 685us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:26.781772 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":281,"threads_started":1}
I20260811 19:02:27.175884 26331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:27.174845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:45339 (local address 127.29.112.130:36187)
0811 19:02:27.175488 (+ 643us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:27.175640 (+ 152us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":345,"threads_started":1}
I20260811 19:02:27.523028 26332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:27.522123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:45339 (local address 127.29.112.129:59629)
0811 19:02:27.522699 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:27.522795 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":289,"threads_started":1}
I20260811 19:02:27.784343 26334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:27.783061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:45339 (local address 127.29.112.131:41831)
0811 19:02:27.783700 (+ 639us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:27.783953 (+ 253us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":305,"threads_started":1}
I20260811 19:02:28.178176 26335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:28.177331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:45339 (local address 127.29.112.130:47687)
0811 19:02:28.177901 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:28.178002 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":311,"threads_started":1}
I20260811 19:02:28.525049 26336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:28.524195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:45339 (local address 127.29.112.129:40433)
0811 19:02:28.524734 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:28.524859 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":295,"threads_started":1}
I20260811 19:02:28.786322 26338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:28.785462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:45339 (local address 127.29.112.131:43561)
0811 19:02:28.786055 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:28.786144 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":338,"threads_started":1}
I20260811 19:02:29.180650 26339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:29.179693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:45339 (local address 127.29.112.130:42887)
0811 19:02:29.180303 (+ 610us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:29.180421 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":324,"threads_started":1}
I20260811 19:02:29.527285 26340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:29.526289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:45339 (local address 127.29.112.129:49341)
0811 19:02:29.526983 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:29.527080 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":576,"thread_start_us":447,"threads_started":1}
I20260811 19:02:29.788522 26341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:29.787644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:45339 (local address 127.29.112.131:47221)
0811 19:02:29.788218 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:29.788321 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":282,"threads_started":1}
I20260811 19:02:30.182439 26342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:30.181696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:45339 (local address 127.29.112.130:38183)
0811 19:02:30.182164 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:30.182248 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":232,"threads_started":1}
I20260811 19:02:30.529534 26344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:30.528669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:45339 (local address 127.29.112.129:47241)
0811 19:02:30.529233 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:30.529344 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":310,"threads_started":1}
I20260811 19:02:30.790218 26345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:30.789484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:45339 (local address 127.29.112.131:40923)
0811 19:02:30.789942 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:30.790036 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":231,"threads_started":1}
I20260811 19:02:31.184820 26346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:31.184010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:45339 (local address 127.29.112.130:54619)
0811 19:02:31.184516 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:31.184634 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":264,"threads_started":1}
I20260811 19:02:31.531793 26347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:31.531030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:45339 (local address 127.29.112.129:48785)
0811 19:02:31.531524 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:31.531606 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":257,"threads_started":1}
I20260811 19:02:31.761638 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 25584
I20260811 19:02:31.791857 26348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:31.763274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40444 (local address 127.29.112.129:46535)
0811 19:02:31.764258 (+ 984us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:31.764282 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:31.764344 (+ 62us) server_negotiation.cc:407] Connection header received
0811 19:02:31.766376 (+ 2032us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:31.766388 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:31.767052 (+ 664us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:31.767395 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:02:31.769516 (+ 2121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:31.771018 (+ 1502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:31.772935 (+ 1917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:31.774134 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:31.774455 (+ 321us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:31.782055 (+ 7600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:02:31.782067 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:02:31.782092 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:02:31.782158 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:02:31.786003 (+ 3845us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:31.787284 (+ 1281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:31.787292 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:31.787300 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:31.787502 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:31.788225 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:31.788229 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:31.788232 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:31.789370 (+ 1138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:02:31.789707 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:31.790626 (+ 919us) server_negotiation.cc:299] Negotiation successful
0811 19:02:31.791417 (+ 791us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":690,"thread_start_us":451,"threads_started":1}
I20260811 19:02:31.793200 26350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:31.791881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:45339 (local address 127.29.112.131:48445)
0811 19:02:31.792764 (+ 883us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:31.792949 (+ 185us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":689,"thread_start_us":502,"threads_started":1}
I20260811 19:02:31.794456 25829 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:02:31.891321 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 25584
I20260811 19:02:31.928476 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 25843
I20260811 19:02:31.953434 26353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:31.929671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33838 (local address 127.29.112.130:37991)
0811 19:02:31.930239 (+ 568us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:31.930254 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:31.930291 (+ 37us) server_negotiation.cc:407] Connection header received
0811 19:02:31.932034 (+ 1743us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:31.932044 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:31.932473 (+ 429us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:31.932768 (+ 295us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:02:31.935259 (+ 2491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:31.936828 (+ 1569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:31.938816 (+ 1988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:31.939577 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:31.940240 (+ 663us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:31.944400 (+ 4160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:02:31.944410 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:02:31.944428 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:02:31.944482 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:02:31.948091 (+ 3609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:31.948973 (+ 882us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:31.948981 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:31.949000 (+ 19us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:31.949209 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:31.950028 (+ 819us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:31.950032 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:31.950034 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:31.950738 (+ 704us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:02:31.951023 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:31.952303 (+ 1280us) server_negotiation.cc:299] Negotiation successful
0811 19:02:31.952957 (+ 654us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":424,"thread_start_us":288,"threads_started":1}
I20260811 19:02:31.955557 26049 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:02:32.046515 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 25843
I20260811 19:02:32.077229 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 26060
I20260811 19:02:32.104797 26362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:32.078824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49372 (local address 127.29.112.131:39661)
0811 19:02:32.079713 (+ 889us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:32.079732 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:32.079791 (+ 59us) server_negotiation.cc:407] Connection header received
0811 19:02:32.081905 (+ 2114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:32.081918 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:32.082801 (+ 883us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:32.083245 (+ 444us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:02:32.085100 (+ 1855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:32.087495 (+ 2395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:32.089407 (+ 1912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:32.090623 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:32.090851 (+ 228us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:32.095567 (+ 4716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:02:32.095581 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:02:32.095608 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:02:32.095675 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:02:32.099987 (+ 4312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:32.101202 (+ 1215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:32.101214 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:32.101222 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:32.101552 (+ 330us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:32.102271 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:32.102277 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:32.102279 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:32.103098 (+ 819us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:02:32.103406 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:32.104004 (+ 598us) server_negotiation.cc:299] Negotiation successful
0811 19:02:32.104437 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":631,"thread_start_us":440,"threads_started":1}
I20260811 19:02:32.106498 26269 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:02:32.206353 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 26060
2026-08-11T19:02:32Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (67910 ms)
[----------] 30 tests from SecurityITest (336965 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:02:32 dist-test-slave-h6n0 krb5kdc[26367](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:02:32 dist-test-slave-h6n0 krb5kdc[26367](info): set up 2 sockets
Aug 11 19:02:32 dist-test-slave-h6n0 krb5kdc[26367](info): commencing operation
krb5kdc: starting...
W20260811 19:02:33.738976 30146 mini_kdc.cc:121] Time spent starting KDC: real 1.479s user 0.000s sys 0.056s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/krb5kdc/test-user.keytab.
Aug 11 19:02:33 dist-test-slave-h6n0 krb5kdc[26367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474953, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:02:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:02:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.112.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab.
Entry for principal kudu/127.29.112.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab.
I20260811 19:02:34.099246 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.190:46783
--webserver_interface=127.29.112.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab
--principal=kudu/127.29.112.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:45495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.190:46783
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:02:34.475687 26392 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:02:34.476294 26392 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:02:34.476529 26392 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:02:34.484709 26392 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:02:34.484836 26392 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:02:34.484959 26392 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:02:34.485059 26392 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:02:34.485138 26392 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:02:34.495184 26392 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:45495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.112.190:46783
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab
--principal=kudu/127.29.112.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.190:46783
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.112.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:02:34.496500 26392 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:02:34.497794 26392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:02:34.510702 26404 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:02:34.510766 26401 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:02:34.511158 26392 server_base.cc:1034] running on GCE node
W20260811 19:02:34.510761 26398 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:02:34.512751 26392 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:02:34.514648 26392 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:02:34.515976 26392 hybrid_clock.cc:648] HybridClock initialized: now 1786474954515924 us; error 54 us; skew 500 ppm
I20260811 19:02:34.516389 26392 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:02:34 dist-test-slave-h6n0 krb5kdc[26367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474954, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:02:34.520138 26392 init.cc:377] Logged in from keytab as kudu/127.29.112.190@KRBTEST.COM (short username kudu)
I20260811 19:02:34.522639 26392 webserver.cc:460] Webserver started at http://127.29.112.190:46255/ using document root <none> and password file <none>
I20260811 19:02:34.523468 26392 fs_manager.cc:359] Metadata directory not provided
I20260811 19:02:34.523636 26392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:02:34.523990 26392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:02:34.527244 26392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/minicluster-data/master-0/data/instance:
uuid: "7708e71da9954be89003ba3ad26de8fe"
format_stamp: "Formatted at 2026-08-11 19:02:34 on dist-test-slave-h6n0"
server_key: "86a9ba9c74c07ba8f720c09cbe875f76"
server_key_iv: "deeb6463f625584aad675bf0784b1c1c"
server_key_version: "encryptionkey@0"
I20260811 19:02:34.528110 26392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/minicluster-data/master-0/wal/instance:
uuid: "7708e71da9954be89003ba3ad26de8fe"
format_stamp: "Formatted at 2026-08-11 19:02:34 on dist-test-slave-h6n0"
server_key: "86a9ba9c74c07ba8f720c09cbe875f76"
server_key_iv: "deeb6463f625584aad675bf0784b1c1c"
server_key_version: "encryptionkey@0"
I20260811 19:02:34.533605 26392 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 19:02:34.538116 26418 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:34.540426 26392 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 19:02:34.540658 26392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/minicluster-data/master-0/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/minicluster-data/master-0/wal
uuid: "7708e71da9954be89003ba3ad26de8fe"
format_stamp: "Formatted at 2026-08-11 19:02:34 on dist-test-slave-h6n0"
server_key: "86a9ba9c74c07ba8f720c09cbe875f76"
server_key_iv: "deeb6463f625584aad675bf0784b1c1c"
server_key_version: "encryptionkey@0"
I20260811 19:02:34.540966 26392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:02:34.556759 26392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:02:34.564903 26392 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:02:34.565366 26392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:02:34.596777 26392 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.190:46783
I20260811 19:02:34.596827 26507 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.190:46783 every 8 connection(s)
I20260811 19:02:34.599069 26392 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
I20260811 19:02:34.604058 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 26392
I20260811 19:02:34.604313 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/minicluster-data/master-0/wal/instance
I20260811 19:02:34.604965 26508 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:02:34.605309 30146 external_mini_cluster.cc:1428] Setting key ac8390b65eea5182dd0aeab694ad755c
I20260811 19:02:34.620538 26508 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7708e71da9954be89003ba3ad26de8fe: Bootstrap starting.
I20260811 19:02:34.623981 26508 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7708e71da9954be89003ba3ad26de8fe: Neither blocks nor log segments found. Creating new log.
I20260811 19:02:34.625094 26508 log.cc:824] T 00000000000000000000000000000000 P 7708e71da9954be89003ba3ad26de8fe: Log is configured to *not* fsync() on all Append() calls
Aug 11 19:02:34 dist-test-slave-h6n0 krb5kdc[26367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474953, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 19:02:34.628082 26508 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7708e71da9954be89003ba3ad26de8fe: No bootstrap required, opened a new log
I20260811 19:02:34.638902 26511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:34.607698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55370 (local address 127.29.112.190:46783)
0811 19:02:34.608595 (+ 897us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:34.608608 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:34.608654 (+ 46us) server_negotiation.cc:407] Connection header received
0811 19:02:34.611973 (+ 3319us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:34.612008 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:34.612593 (+ 585us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:34.615339 (+ 2746us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:02:34.615943 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:34.618505 (+ 2562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:34.620813 (+ 2308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:34.622178 (+ 1365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:34.623233 (+ 1055us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:34.628767 (+ 5534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:02:34.628780 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:02:34.628801 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:02:34.628857 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:02:34.633455 (+ 4598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:34.634506 (+ 1051us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:34.634515 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:34.634524 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:34.634777 (+ 253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:34.635303 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:34.635308 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:34.635313 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:34.636099 (+ 786us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:02:34.636410 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:34.637008 (+ 598us) server_negotiation.cc:299] Negotiation successful
0811 19:02:34.637489 (+ 481us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":663,"thread_start_us":382,"threads_started":1}
I20260811 19:02:34.639437 26508 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7708e71da9954be89003ba3ad26de8fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7708e71da9954be89003ba3ad26de8fe" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 46783 } }
I20260811 19:02:34.639935 26508 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7708e71da9954be89003ba3ad26de8fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:02:34.640167 26508 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7708e71da9954be89003ba3ad26de8fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7708e71da9954be89003ba3ad26de8fe, State: Initialized, Role: FOLLOWER
I20260811 19:02:34.640741 26508 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7708e71da9954be89003ba3ad26de8fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7708e71da9954be89003ba3ad26de8fe" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 46783 } }
I20260811 19:02:34.641002 26508 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7708e71da9954be89003ba3ad26de8fe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:02:34.641196 26508 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7708e71da9954be89003ba3ad26de8fe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:02:34.641386 26508 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7708e71da9954be89003ba3ad26de8fe [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:02:34.643956 26508 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7708e71da9954be89003ba3ad26de8fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7708e71da9954be89003ba3ad26de8fe" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 46783 } }
I20260811 19:02:34.644500 26508 leader_election.cc:302] T 00000000000000000000000000000000 P 7708e71da9954be89003ba3ad26de8fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7708e71da9954be89003ba3ad26de8fe; no voters:
I20260811 19:02:34.645241 26508 leader_election.cc:288] T 00000000000000000000000000000000 P 7708e71da9954be89003ba3ad26de8fe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:02:34.645387 26513 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7708e71da9954be89003ba3ad26de8fe [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:02:34.647830 26508 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7708e71da9954be89003ba3ad26de8fe [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:02:34.651000 26513 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7708e71da9954be89003ba3ad26de8fe [term 1 LEADER]: Becoming Leader. State: Replica: 7708e71da9954be89003ba3ad26de8fe, State: Running, Role: LEADER
I20260811 19:02:34.651933 26513 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7708e71da9954be89003ba3ad26de8fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7708e71da9954be89003ba3ad26de8fe" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 46783 } }
I20260811 19:02:34.655889 26514 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7708e71da9954be89003ba3ad26de8fe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7708e71da9954be89003ba3ad26de8fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7708e71da9954be89003ba3ad26de8fe" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 46783 } } }
I20260811 19:02:34.656364 26514 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7708e71da9954be89003ba3ad26de8fe [sys.catalog]: This master's current role is: LEADER
I20260811 19:02:34.658119 26516 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7708e71da9954be89003ba3ad26de8fe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7708e71da9954be89003ba3ad26de8fe. Latest consensus state: current_term: 1 leader_uuid: "7708e71da9954be89003ba3ad26de8fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7708e71da9954be89003ba3ad26de8fe" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 46783 } } }
I20260811 19:02:34.658438 26516 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7708e71da9954be89003ba3ad26de8fe [sys.catalog]: This master's current role is: LEADER
I20260811 19:02:34.659801 26527 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:02:34.665630 26527 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:02:34.688630 26527 catalog_manager.cc:1347] Generated new cluster ID: a7fd276447ed4797ac64c0f16f0b0253
I20260811 19:02:34.688848 26527 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:02:34.729419 26527 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:02:34.730937 26527 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:02:34.741586 26527 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7708e71da9954be89003ba3ad26de8fe: Generated new TSK 0
I20260811 19:02:34.742519 26527 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.112.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.129.keytab.
Entry for principal kudu/127.29.112.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.129.keytab.
I20260811 19:02:34.851194 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.129:0
--local_ip_for_outbound_sockets=127.29.112.129
--webserver_interface=127.29.112.129
--webserver_port=0
--tserver_master_addrs=127.29.112.190:46783
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.129.keytab
--principal=kudu/127.29.112.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:45495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:02:35.235703 26546 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:02:35.235973 26546 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:02:35.236131 26546 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:02:35.243723 26546 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:02:35.243875 26546 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:02:35.244045 26546 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.129
I20260811 19:02:35.254565 26546 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:45495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.129.keytab
--principal=kudu/127.29.112.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.112.129
--webserver_port=0
--tserver_master_addrs=127.29.112.190:46783
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.129
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:02:35.256172 26546 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:02:35.257565 26546 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:02:35.273698 26554 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:02:35.273734 26553 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:02:35.273663 26561 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:02:35.275619 26546 server_base.cc:1034] running on GCE node
I20260811 19:02:35.276377 26546 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:02:35.277562 26546 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:02:35.281234 26546 hybrid_clock.cc:648] HybridClock initialized: now 1786474955281199 us; error 96 us; skew 500 ppm
I20260811 19:02:35.282163 26546 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:02:35 dist-test-slave-h6n0 krb5kdc[26367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474955, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:02:35.287137 26546 init.cc:377] Logged in from keytab as kudu/127.29.112.129@KRBTEST.COM (short username kudu)
I20260811 19:02:35.289536 26546 webserver.cc:460] Webserver started at http://127.29.112.129:40941/ using document root <none> and password file <none>
I20260811 19:02:35.290324 26546 fs_manager.cc:359] Metadata directory not provided
I20260811 19:02:35.290463 26546 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:02:35.290874 26546 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:02:35.295102 26546 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/minicluster-data/ts-0/data/instance:
uuid: "f4489642373b40788f9991efef9212bd"
format_stamp: "Formatted at 2026-08-11 19:02:35 on dist-test-slave-h6n0"
server_key: "6a84525b522c1c0195be9f56d133ad83"
server_key_iv: "e5d4943a8d48166d2229ad3aeb729fde"
server_key_version: "encryptionkey@0"
I20260811 19:02:35.296020 26546 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/minicluster-data/ts-0/wal/instance:
uuid: "f4489642373b40788f9991efef9212bd"
format_stamp: "Formatted at 2026-08-11 19:02:35 on dist-test-slave-h6n0"
server_key: "6a84525b522c1c0195be9f56d133ad83"
server_key_iv: "e5d4943a8d48166d2229ad3aeb729fde"
server_key_version: "encryptionkey@0"
I20260811 19:02:35.302446 26546 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260811 19:02:35.308136 26571 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:35.311123 26546 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.006s sys 0.000s
I20260811 19:02:35.311381 26546 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/minicluster-data/ts-0/wal
uuid: "f4489642373b40788f9991efef9212bd"
format_stamp: "Formatted at 2026-08-11 19:02:35 on dist-test-slave-h6n0"
server_key: "6a84525b522c1c0195be9f56d133ad83"
server_key_iv: "e5d4943a8d48166d2229ad3aeb729fde"
server_key_version: "encryptionkey@0"
I20260811 19:02:35.311656 26546 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:02:35.341161 26546 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:02:35.347659 26546 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:02:35.348086 26546 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:02:35.349363 26546 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:02:35.351060 26546 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:02:35.351176 26546 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:35.351356 26546 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:02:35.351441 26546 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:35.398747 26546 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.129:37705
I20260811 19:02:35.398830 26752 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.129:37705 every 8 connection(s)
I20260811 19:02:35.400779 26546 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/minicluster-data/ts-0/data/info.pb
I20260811 19:02:35.409407 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 26546
I20260811 19:02:35.409678 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/minicluster-data/ts-0/wal/instance
I20260811 19:02:35.410517 30146 external_mini_cluster.cc:1428] Setting key 40ae78717806362bbf94b57cfb1987a9
Aug 11 19:02:35 dist-test-slave-h6n0 krb5kdc[26367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474955, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.129@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 19:02:35.430099 26756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:35.404079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.129:35269 (local address 127.29.112.190:46783)
0811 19:02:35.404674 (+ 595us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:35.404683 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:35.405732 (+ 1049us) server_negotiation.cc:407] Connection header received
0811 19:02:35.407775 (+ 2043us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:35.407780 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:35.407854 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:35.407979 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:02:35.410302 (+ 2323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:35.411917 (+ 1615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:35.414387 (+ 2470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:35.415156 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:35.415344 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:35.420982 (+ 5638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:02:35.420992 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:02:35.421000 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:02:35.421056 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:02:35.425403 (+ 4347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:35.426631 (+ 1228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:35.426640 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:35.426647 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:35.426825 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:35.427473 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:35.427481 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:35.427488 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:35.427949 (+ 461us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:02:35.428153 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:35.429564 (+ 1411us) server_negotiation.cc:299] Negotiation successful
0811 19:02:35.429764 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":451,"thread_start_us":345,"threads_started":1}
I20260811 19:02:35.431712 26758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:35.404727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:46783 (local address 127.29.112.129:35269)
0811 19:02:35.405545 (+ 818us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:35.405589 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 19:02:35.407480 (+ 1891us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:02:35.408251 (+ 771us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:02:35.408260 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:02:35.408995 (+ 735us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:02:35.409976 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:35.410006 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:35.412182 (+ 2176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:35.412191 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:35.414030 (+ 1839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:35.414046 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:35.415634 (+ 1588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:35.415643 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:35.415988 (+ 345us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:35.417083 (+ 1095us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:02:35.417115 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:02:35.420683 (+ 3568us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:02:35.425633 (+ 4950us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:02:35.425643 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:02:35.425659 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:02:35.426446 (+ 787us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:02:35.427038 (+ 592us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:02:35.427043 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:02:35.427046 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:02:35.427291 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:02:35.428172 (+ 881us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:02:35.428179 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:02:35.428741 (+ 562us) client_negotiation.cc:769] Sending connection context
0811 19:02:35.429432 (+ 691us) client_negotiation.cc:240] Negotiation successful
0811 19:02:35.430106 (+ 674us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":561,"thread_start_us":315,"threads_started":1}
I20260811 19:02:35.434077 26753 heartbeater.cc:347] Connected to a master server at 127.29.112.190:46783
I20260811 19:02:35.434463 26753 heartbeater.cc:464] Registering TS with master...
I20260811 19:02:35.435295 26753 heartbeater.cc:511] Master 127.29.112.190:46783 requested a full tablet report, sending...
I20260811 19:02:35.437937 26453 ts_manager.cc:194] Registered new tserver with Master: f4489642373b40788f9991efef9212bd (127.29.112.129:37705)
I20260811 19:02:35.440438 26453 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.129@KRBTEST.COM'} at 127.29.112.129:35269
WARNING: no policy specified for kudu/127.29.112.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.130.keytab.
Entry for principal kudu/127.29.112.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.130.keytab.
I20260811 19:02:35.494382 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.130:0
--local_ip_for_outbound_sockets=127.29.112.130
--webserver_interface=127.29.112.130
--webserver_port=0
--tserver_master_addrs=127.29.112.190:46783
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.130.keytab
--principal=kudu/127.29.112.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:45495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:02:35.877463 26766 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:02:35.877913 26766 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:02:35.878031 26766 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:02:35.888047 26766 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:02:35.888234 26766 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:02:35.888482 26766 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.130
I20260811 19:02:35.899510 26766 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:45495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.130.keytab
--principal=kudu/127.29.112.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.112.130
--webserver_port=0
--tserver_master_addrs=127.29.112.190:46783
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.130
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:02:35.900808 26766 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:02:35.902100 26766 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:02:35.916651 26775 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:02:35.916719 26780 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:02:35.916618 26778 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:02:35.918298 26766 server_base.cc:1034] running on GCE node
I20260811 19:02:35.919695 26766 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:02:35.920866 26766 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:02:35.922250 26766 hybrid_clock.cc:648] HybridClock initialized: now 1786474955922168 us; error 51 us; skew 500 ppm
I20260811 19:02:35.922976 26766 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:02:35 dist-test-slave-h6n0 krb5kdc[26367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474955, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:02:35.926834 26766 init.cc:377] Logged in from keytab as kudu/127.29.112.130@KRBTEST.COM (short username kudu)
I20260811 19:02:35.928975 26766 webserver.cc:460] Webserver started at http://127.29.112.130:34677/ using document root <none> and password file <none>
I20260811 19:02:35.929792 26766 fs_manager.cc:359] Metadata directory not provided
I20260811 19:02:35.929976 26766 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:02:35.930367 26766 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:02:35.933744 26766 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/minicluster-data/ts-1/data/instance:
uuid: "5cb7a19aec52490a8d8229304f7776fb"
format_stamp: "Formatted at 2026-08-11 19:02:35 on dist-test-slave-h6n0"
server_key: "b5489718472daa9e4c44b972eeb0e3b0"
server_key_iv: "c97b80eed8eb344afe0dac2bf67f6ba8"
server_key_version: "encryptionkey@0"
I20260811 19:02:35.934736 26766 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/minicluster-data/ts-1/wal/instance:
uuid: "5cb7a19aec52490a8d8229304f7776fb"
format_stamp: "Formatted at 2026-08-11 19:02:35 on dist-test-slave-h6n0"
server_key: "b5489718472daa9e4c44b972eeb0e3b0"
server_key_iv: "c97b80eed8eb344afe0dac2bf67f6ba8"
server_key_version: "encryptionkey@0"
I20260811 19:02:35.940572 26766 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 19:02:35.944585 26795 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:35.946244 26766 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 19:02:35.946446 26766 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/minicluster-data/ts-1/wal
uuid: "5cb7a19aec52490a8d8229304f7776fb"
format_stamp: "Formatted at 2026-08-11 19:02:35 on dist-test-slave-h6n0"
server_key: "b5489718472daa9e4c44b972eeb0e3b0"
server_key_iv: "c97b80eed8eb344afe0dac2bf67f6ba8"
server_key_version: "encryptionkey@0"
I20260811 19:02:35.946795 26766 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:02:36.007472 26766 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:02:36.012888 26766 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:02:36.013258 26766 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:02:36.014575 26766 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:02:36.016203 26766 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:02:36.016589 26766 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260811 19:02:36.016880 26766 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:02:36.016940 26766 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:36.058868 26766 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.130:46519
I20260811 19:02:36.058924 26978 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.130:46519 every 8 connection(s)
I20260811 19:02:36.060221 26766 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/minicluster-data/ts-1/data/info.pb
I20260811 19:02:36.063648 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 26766
I20260811 19:02:36.063895 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/minicluster-data/ts-1/wal/instance
I20260811 19:02:36.064579 30146 external_mini_cluster.cc:1428] Setting key 9f62bd326d0780b4666e9358c49ac99a
Aug 11 19:02:36 dist-test-slave-h6n0 krb5kdc[26367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474955, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.130@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 19:02:36.084475 26984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:36.063184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.130:35147 (local address 127.29.112.190:46783)
0811 19:02:36.063800 (+ 616us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:36.063810 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:36.064860 (+ 1050us) server_negotiation.cc:407] Connection header received
0811 19:02:36.067858 (+ 2998us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:36.067866 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:36.067971 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:36.068131 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:02:36.069899 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:36.071090 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:36.073375 (+ 2285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:36.073860 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:36.074011 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:36.077903 (+ 3892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:02:36.077908 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:02:36.077911 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:02:36.077945 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:02:36.081283 (+ 3338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:36.082057 (+ 774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:36.082064 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:36.082067 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:36.082198 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:36.082685 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:36.082690 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:36.082694 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:36.082980 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:02:36.083161 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:36.083860 (+ 699us) server_negotiation.cc:299] Negotiation successful
0811 19:02:36.084084 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":473,"thread_start_us":345,"threads_started":1}
I20260811 19:02:36.085420 26985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:36.063795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:46783 (local address 127.29.112.130:35147)
0811 19:02:36.064669 (+ 874us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:36.064734 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0811 19:02:36.067312 (+ 2578us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:02:36.068401 (+ 1089us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:02:36.068411 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:02:36.068978 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:02:36.069555 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:36.069574 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:36.071304 (+ 1730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:36.071309 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:36.073178 (+ 1869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:36.073195 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:36.074114 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:36.074121 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:36.074360 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:36.075169 (+ 809us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:02:36.075194 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:02:36.077656 (+ 2462us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:02:36.081453 (+ 3797us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:02:36.081463 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:02:36.081482 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:02:36.081929 (+ 447us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:02:36.082311 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:02:36.082316 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:02:36.082319 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:02:36.082562 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:02:36.083129 (+ 567us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:02:36.083137 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:02:36.083478 (+ 341us) client_negotiation.cc:769] Sending connection context
0811 19:02:36.083767 (+ 289us) client_negotiation.cc:240] Negotiation successful
0811 19:02:36.084318 (+ 551us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":594,"thread_start_us":354,"threads_started":1}
I20260811 19:02:36.087141 26981 heartbeater.cc:347] Connected to a master server at 127.29.112.190:46783
I20260811 19:02:36.087493 26981 heartbeater.cc:464] Registering TS with master...
I20260811 19:02:36.088377 26981 heartbeater.cc:511] Master 127.29.112.190:46783 requested a full tablet report, sending...
I20260811 19:02:36.090022 26453 ts_manager.cc:194] Registered new tserver with Master: 5cb7a19aec52490a8d8229304f7776fb (127.29.112.130:46519)
I20260811 19:02:36.091007 26453 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.130@KRBTEST.COM'} at 127.29.112.130:35147
WARNING: no policy specified for kudu/127.29.112.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.131.keytab.
Entry for principal kudu/127.29.112.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.131.keytab.
I20260811 19:02:36.151012 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.131:0
--local_ip_for_outbound_sockets=127.29.112.131
--webserver_interface=127.29.112.131
--webserver_port=0
--tserver_master_addrs=127.29.112.190:46783
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.131.keytab
--principal=kudu/127.29.112.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:45495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 19:02:36.444408 26753 heartbeater.cc:503] Master 127.29.112.190:46783 was elected leader, sending a full tablet report...
W20260811 19:02:36.530625 26994 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:02:36.530987 26994 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:02:36.531162 26994 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:02:36.538465 26994 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:02:36.538664 26994 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:02:36.538857 26994 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.131
I20260811 19:02:36.549990 26994 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:45495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.131.keytab
--principal=kudu/127.29.112.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.112.131
--webserver_port=0
--tserver_master_addrs=127.29.112.190:46783
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.131
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:02:36.551554 26994 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:02:36.552990 26994 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:02:36.568879 27004 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:02:36.569175 27005 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:02:36.569286 27008 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:02:36.571035 26994 server_base.cc:1034] running on GCE node
I20260811 19:02:36.571857 26994 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:02:36.573149 26994 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:02:36.574568 26994 hybrid_clock.cc:648] HybridClock initialized: now 1786474956574440 us; error 60 us; skew 500 ppm
I20260811 19:02:36.574942 26994 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:02:36 dist-test-slave-h6n0 krb5kdc[26367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474956, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:02:36.579463 26994 init.cc:377] Logged in from keytab as kudu/127.29.112.131@KRBTEST.COM (short username kudu)
I20260811 19:02:36.581072 26994 webserver.cc:460] Webserver started at http://127.29.112.131:38581/ using document root <none> and password file <none>
I20260811 19:02:36.581753 26994 fs_manager.cc:359] Metadata directory not provided
I20260811 19:02:36.581856 26994 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:02:36.582115 26994 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:02:36.585253 26994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/minicluster-data/ts-2/data/instance:
uuid: "9382c1e24e04478c834ce2433b5ec413"
format_stamp: "Formatted at 2026-08-11 19:02:36 on dist-test-slave-h6n0"
server_key: "cac50f36e62e6f3bcd5d40a209790b0e"
server_key_iv: "5b5431b79b5851954fa068c211c47f26"
server_key_version: "encryptionkey@0"
I20260811 19:02:36.585991 26994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/minicluster-data/ts-2/wal/instance:
uuid: "9382c1e24e04478c834ce2433b5ec413"
format_stamp: "Formatted at 2026-08-11 19:02:36 on dist-test-slave-h6n0"
server_key: "cac50f36e62e6f3bcd5d40a209790b0e"
server_key_iv: "5b5431b79b5851954fa068c211c47f26"
server_key_version: "encryptionkey@0"
I20260811 19:02:36.592198 26994 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260811 19:02:36.596797 27021 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:36.598733 26994 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260811 19:02:36.598927 26994 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/minicluster-data/ts-2/wal
uuid: "9382c1e24e04478c834ce2433b5ec413"
format_stamp: "Formatted at 2026-08-11 19:02:36 on dist-test-slave-h6n0"
server_key: "cac50f36e62e6f3bcd5d40a209790b0e"
server_key_iv: "5b5431b79b5851954fa068c211c47f26"
server_key_version: "encryptionkey@0"
I20260811 19:02:36.599149 26994 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:02:36.645288 26994 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:02:36.650350 26994 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:02:36.650733 26994 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:02:36.651877 26994 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:02:36.653406 26994 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:02:36.653486 26994 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:36.653625 26994 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:02:36.653685 26994 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:36.691763 26994 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.131:36761
I20260811 19:02:36.692167 27219 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.131:36761 every 8 connection(s)
I20260811 19:02:36.693173 26994 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/minicluster-data/ts-2/data/info.pb
I20260811 19:02:36.697504 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 26994
I20260811 19:02:36.697733 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/minicluster-data/ts-2/wal/instance
I20260811 19:02:36.698354 30146 external_mini_cluster.cc:1428] Setting key e0ef251ccc044511e7776a8823532124
Aug 11 19:02:36 dist-test-slave-h6n0 krb5kdc[26367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474956, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.131@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 19:02:36.717649 27223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:36.697113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.131:49455 (local address 127.29.112.190:46783)
0811 19:02:36.697677 (+ 564us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:36.697686 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:36.700408 (+ 2722us) server_negotiation.cc:407] Connection header received
0811 19:02:36.700706 (+ 298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:36.700712 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:36.700812 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:36.700939 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:02:36.702620 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:36.703400 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:36.705133 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:36.705560 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:36.705710 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:36.709790 (+ 4080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:02:36.709798 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:02:36.709802 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:02:36.709858 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:02:36.714203 (+ 4345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:36.715067 (+ 864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:36.715075 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:36.715078 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:36.715219 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:36.715679 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:36.715684 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:36.715692 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:36.715965 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:02:36.716142 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:36.716959 (+ 817us) server_negotiation.cc:299] Negotiation successful
0811 19:02:36.717200 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":414,"thread_start_us":267,"threads_started":1}
I20260811 19:02:36.718741 27225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:36.697406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:46783 (local address 127.29.112.131:49455)
0811 19:02:36.698339 (+ 933us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:36.698396 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 19:02:36.700425 (+ 2029us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:02:36.701185 (+ 760us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:02:36.701192 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:02:36.701839 (+ 647us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:02:36.702418 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:36.702435 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:36.703553 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:36.703563 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:36.704986 (+ 1423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:36.704999 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:36.705794 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:36.705802 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:36.706045 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:36.706802 (+ 757us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:02:36.706825 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:02:36.709246 (+ 2421us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:02:36.714363 (+ 5117us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:02:36.714381 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:02:36.714395 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:02:36.714940 (+ 545us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:02:36.715345 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:02:36.715352 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:02:36.715361 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:02:36.715550 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:02:36.716105 (+ 555us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:02:36.716112 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:02:36.716520 (+ 408us) client_negotiation.cc:769] Sending connection context
0811 19:02:36.716856 (+ 336us) client_negotiation.cc:240] Negotiation successful
0811 19:02:36.717422 (+ 566us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":641,"thread_start_us":400,"threads_started":1}
I20260811 19:02:36.720466 27220 heartbeater.cc:347] Connected to a master server at 127.29.112.190:46783
I20260811 19:02:36.720779 27220 heartbeater.cc:464] Registering TS with master...
I20260811 19:02:36.721553 27220 heartbeater.cc:511] Master 127.29.112.190:46783 requested a full tablet report, sending...
I20260811 19:02:36.723578 26453 ts_manager.cc:194] Registered new tserver with Master: 9382c1e24e04478c834ce2433b5ec413 (127.29.112.131:36761)
I20260811 19:02:36.724516 26453 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.131@KRBTEST.COM'} at 127.29.112.131:49455
I20260811 19:02:36.727880 30146 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 19:02:36.749892 27223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:36.733009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55374 (local address 127.29.112.190:46783)
0811 19:02:36.733434 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:36.733442 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:36.733580 (+ 138us) server_negotiation.cc:407] Connection header received
0811 19:02:36.733964 (+ 384us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:36.733970 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:36.734126 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:36.734376 (+ 250us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:02:36.736534 (+ 2158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:36.737243 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:36.739254 (+ 2011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:36.739854 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:36.740001 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:36.743204 (+ 3203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:02:36.743210 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:02:36.743214 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:02:36.743261 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:02:36.746660 (+ 3399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:36.747537 (+ 877us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:36.747542 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:36.747544 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:36.747683 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:36.748216 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:36.748220 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:36.748223 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:36.748652 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:02:36.748861 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:36.749300 (+ 439us) server_negotiation.cc:299] Negotiation successful
0811 19:02:36.749501 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136}
I20260811 19:02:36.756081 26453 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55374:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 19:02:36.763572 26453 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 19:02:36.802062 27254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:36.778228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40948 (local address 127.29.112.129:37705)
0811 19:02:36.786413 (+ 8185us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:36.786428 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:36.791672 (+ 5244us) server_negotiation.cc:407] Connection header received
0811 19:02:36.792554 (+ 882us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:36.792568 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:36.792926 (+ 358us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:36.793174 (+ 248us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:02:36.794348 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:36.796413 (+ 2065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:36.799328 (+ 2915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:36.800817 (+ 1489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:36.801036 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:36.801232 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:36.801525 (+ 293us) server_negotiation.cc:299] Negotiation successful
0811 19:02:36.801692 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7941,"thread_start_us":7693,"threads_started":1}
I20260811 19:02:36.802114 27255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:36.777198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:37705 (local address 127.0.0.1:40948)
0811 19:02:36.791483 (+ 14285us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:36.791517 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0811 19:02:36.792281 (+ 764us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:02:36.793592 (+ 1311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:02:36.793603 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:02:36.793648 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:02:36.794124 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:36.794137 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:36.796605 (+ 2468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:36.796613 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:36.799165 (+ 2552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:36.799177 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:36.801042 (+ 1865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:36.801048 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:36.801246 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:36.801300 (+ 54us) client_negotiation.cc:769] Sending connection context
0811 19:02:36.801482 (+ 182us) client_negotiation.cc:240] Negotiation successful
0811 19:02:36.801683 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14098,"mutex_wait_us":1,"thread_start_us":17514,"threads_started":1}
I20260811 19:02:36.803843 27245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:36.776791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.131:36761 (local address 127.0.0.1:41356)
0811 19:02:36.778298 (+ 1507us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:36.778346 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 19:02:36.778548 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:02:36.795888 (+ 17340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:02:36.795893 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:02:36.795930 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:02:36.796665 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:36.796681 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:36.798396 (+ 1715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:36.798404 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:36.800894 (+ 2490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:36.800918 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:36.802974 (+ 2056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:36.802980 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:36.803156 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:36.803187 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 19:02:36.803304 (+ 117us) client_negotiation.cc:240] Negotiation successful
0811 19:02:36.803435 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1265,"thread_start_us":1108,"threads_started":1}
I20260811 19:02:36.805096 27248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:36.778734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.130:46519 (local address 127.0.0.1:56990)
0811 19:02:36.794997 (+ 16263us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:36.795022 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 19:02:36.795212 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:02:36.796157 (+ 945us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:02:36.796162 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:02:36.796192 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:02:36.797112 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:36.797128 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:36.799033 (+ 1905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:36.799042 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:36.801539 (+ 2497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:36.801556 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:36.804257 (+ 2701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:36.804268 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:36.804451 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:36.804486 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 19:02:36.804595 (+ 109us) client_negotiation.cc:240] Negotiation successful
0811 19:02:36.804746 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16145,"thread_start_us":12982,"threads_started":1}
I20260811 19:02:36.805923 27258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:36.779110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56990 (local address 127.29.112.130:46519)
0811 19:02:36.794339 (+ 15229us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:36.794348 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:36.795116 (+ 768us) server_negotiation.cc:407] Connection header received
0811 19:02:36.795413 (+ 297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:36.795424 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:36.795637 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:36.795857 (+ 220us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:02:36.797329 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:36.798881 (+ 1552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:36.801747 (+ 2866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:36.803573 (+ 1826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:36.803757 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:36.805320 (+ 1563us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:36.805396 (+ 76us) server_negotiation.cc:299] Negotiation successful
0811 19:02:36.805584 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15027,"thread_start_us":15239,"threads_started":1}
I20260811 19:02:36.806818 27247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:36.777215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41356 (local address 127.29.112.131:36761)
0811 19:02:36.794911 (+ 17696us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:36.794920 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:36.794958 (+ 38us) server_negotiation.cc:407] Connection header received
0811 19:02:36.795067 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:36.795075 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:36.795639 (+ 564us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:36.796771 (+ 1132us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:02:36.797078 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:36.798254 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:36.801133 (+ 2879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:36.802778 (+ 1645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:36.802983 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:36.803172 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:36.803458 (+ 286us) server_negotiation.cc:299] Negotiation successful
0811 19:02:36.806364 (+ 2906us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17492,"thread_start_us":17720,"threads_started":1}
I20260811 19:02:36.807093 26656 tablet_service.cc:1467] Processing CreateTablet for tablet 5363877ee27a42709bdf1c5e8f3ce39b (DEFAULT_TABLE table=test-table [id=6a83f591f58e41dfb415f1902dbdd3fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:02:36.808651 26656 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5363877ee27a42709bdf1c5e8f3ce39b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:02:36.809830 26870 tablet_service.cc:1467] Processing CreateTablet for tablet 5363877ee27a42709bdf1c5e8f3ce39b (DEFAULT_TABLE table=test-table [id=6a83f591f58e41dfb415f1902dbdd3fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:02:36.811295 26870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5363877ee27a42709bdf1c5e8f3ce39b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:02:36.810837 27108 tablet_service.cc:1467] Processing CreateTablet for tablet 5363877ee27a42709bdf1c5e8f3ce39b (DEFAULT_TABLE table=test-table [id=6a83f591f58e41dfb415f1902dbdd3fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:02:36.812157 27108 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5363877ee27a42709bdf1c5e8f3ce39b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:02:36.825521 27260 tablet_bootstrap.cc:492] T 5363877ee27a42709bdf1c5e8f3ce39b P f4489642373b40788f9991efef9212bd: Bootstrap starting.
I20260811 19:02:36.827780 27265 tablet_bootstrap.cc:492] T 5363877ee27a42709bdf1c5e8f3ce39b P 9382c1e24e04478c834ce2433b5ec413: Bootstrap starting.
I20260811 19:02:36.830466 27260 tablet_bootstrap.cc:654] T 5363877ee27a42709bdf1c5e8f3ce39b P f4489642373b40788f9991efef9212bd: Neither blocks nor log segments found. Creating new log.
I20260811 19:02:36.831553 27265 tablet_bootstrap.cc:654] T 5363877ee27a42709bdf1c5e8f3ce39b P 9382c1e24e04478c834ce2433b5ec413: Neither blocks nor log segments found. Creating new log.
I20260811 19:02:36.832149 27260 log.cc:824] T 5363877ee27a42709bdf1c5e8f3ce39b P f4489642373b40788f9991efef9212bd: Log is configured to *not* fsync() on all Append() calls
I20260811 19:02:36.832954 27265 log.cc:824] T 5363877ee27a42709bdf1c5e8f3ce39b P 9382c1e24e04478c834ce2433b5ec413: Log is configured to *not* fsync() on all Append() calls
I20260811 19:02:36.834573 27263 tablet_bootstrap.cc:492] T 5363877ee27a42709bdf1c5e8f3ce39b P 5cb7a19aec52490a8d8229304f7776fb: Bootstrap starting.
I20260811 19:02:36.839568 27263 tablet_bootstrap.cc:654] T 5363877ee27a42709bdf1c5e8f3ce39b P 5cb7a19aec52490a8d8229304f7776fb: Neither blocks nor log segments found. Creating new log.
I20260811 19:02:36.840795 27263 log.cc:824] T 5363877ee27a42709bdf1c5e8f3ce39b P 5cb7a19aec52490a8d8229304f7776fb: Log is configured to *not* fsync() on all Append() calls
I20260811 19:02:36.841347 27265 tablet_bootstrap.cc:492] T 5363877ee27a42709bdf1c5e8f3ce39b P 9382c1e24e04478c834ce2433b5ec413: No bootstrap required, opened a new log
I20260811 19:02:36.841678 27265 ts_tablet_manager.cc:1397] T 5363877ee27a42709bdf1c5e8f3ce39b P 9382c1e24e04478c834ce2433b5ec413: Time spent bootstrapping tablet: real 0.014s user 0.006s sys 0.000s
I20260811 19:02:36.841894 27260 tablet_bootstrap.cc:492] T 5363877ee27a42709bdf1c5e8f3ce39b P f4489642373b40788f9991efef9212bd: No bootstrap required, opened a new log
I20260811 19:02:36.842175 27260 ts_tablet_manager.cc:1397] T 5363877ee27a42709bdf1c5e8f3ce39b P f4489642373b40788f9991efef9212bd: Time spent bootstrapping tablet: real 0.017s user 0.007s sys 0.007s
I20260811 19:02:36.844256 27263 tablet_bootstrap.cc:492] T 5363877ee27a42709bdf1c5e8f3ce39b P 5cb7a19aec52490a8d8229304f7776fb: No bootstrap required, opened a new log
I20260811 19:02:36.844597 27263 ts_tablet_manager.cc:1397] T 5363877ee27a42709bdf1c5e8f3ce39b P 5cb7a19aec52490a8d8229304f7776fb: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.004s
I20260811 19:02:36.852092 27265 raft_consensus.cc:348] T 5363877ee27a42709bdf1c5e8f3ce39b P 9382c1e24e04478c834ce2433b5ec413 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9382c1e24e04478c834ce2433b5ec413" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 36761 } } peers { permanent_uuid: "f4489642373b40788f9991efef9212bd" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 37705 } } peers { permanent_uuid: "5cb7a19aec52490a8d8229304f7776fb" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 46519 } }
I20260811 19:02:36.852591 27265 raft_consensus.cc:374] T 5363877ee27a42709bdf1c5e8f3ce39b P 9382c1e24e04478c834ce2433b5ec413 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:02:36.852708 27265 raft_consensus.cc:729] T 5363877ee27a42709bdf1c5e8f3ce39b P 9382c1e24e04478c834ce2433b5ec413 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9382c1e24e04478c834ce2433b5ec413, State: Initialized, Role: FOLLOWER
I20260811 19:02:36.853200 27260 raft_consensus.cc:348] T 5363877ee27a42709bdf1c5e8f3ce39b P f4489642373b40788f9991efef9212bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9382c1e24e04478c834ce2433b5ec413" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 36761 } } peers { permanent_uuid: "f4489642373b40788f9991efef9212bd" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 37705 } } peers { permanent_uuid: "5cb7a19aec52490a8d8229304f7776fb" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 46519 } }
I20260811 19:02:36.853400 27265 consensus_queue.cc:260] T 5363877ee27a42709bdf1c5e8f3ce39b P 9382c1e24e04478c834ce2433b5ec413 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9382c1e24e04478c834ce2433b5ec413" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 36761 } } peers { permanent_uuid: "f4489642373b40788f9991efef9212bd" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 37705 } } peers { permanent_uuid: "5cb7a19aec52490a8d8229304f7776fb" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 46519 } }
I20260811 19:02:36.853631 27260 raft_consensus.cc:374] T 5363877ee27a42709bdf1c5e8f3ce39b P f4489642373b40788f9991efef9212bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:02:36.853731 27260 raft_consensus.cc:729] T 5363877ee27a42709bdf1c5e8f3ce39b P f4489642373b40788f9991efef9212bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4489642373b40788f9991efef9212bd, State: Initialized, Role: FOLLOWER
I20260811 19:02:36.854689 27220 heartbeater.cc:503] Master 127.29.112.190:46783 was elected leader, sending a full tablet report...
I20260811 19:02:36.854969 27260 consensus_queue.cc:260] T 5363877ee27a42709bdf1c5e8f3ce39b P f4489642373b40788f9991efef9212bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9382c1e24e04478c834ce2433b5ec413" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 36761 } } peers { permanent_uuid: "f4489642373b40788f9991efef9212bd" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 37705 } } peers { permanent_uuid: "5cb7a19aec52490a8d8229304f7776fb" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 46519 } }
I20260811 19:02:36.855523 27265 ts_tablet_manager.cc:1428] T 5363877ee27a42709bdf1c5e8f3ce39b P 9382c1e24e04478c834ce2433b5ec413: Time spent starting tablet: real 0.014s user 0.015s sys 0.000s
I20260811 19:02:36.856244 27263 raft_consensus.cc:348] T 5363877ee27a42709bdf1c5e8f3ce39b P 5cb7a19aec52490a8d8229304f7776fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9382c1e24e04478c834ce2433b5ec413" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 36761 } } peers { permanent_uuid: "f4489642373b40788f9991efef9212bd" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 37705 } } peers { permanent_uuid: "5cb7a19aec52490a8d8229304f7776fb" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 46519 } }
I20260811 19:02:36.856686 27263 raft_consensus.cc:374] T 5363877ee27a42709bdf1c5e8f3ce39b P 5cb7a19aec52490a8d8229304f7776fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:02:36.856804 27263 raft_consensus.cc:729] T 5363877ee27a42709bdf1c5e8f3ce39b P 5cb7a19aec52490a8d8229304f7776fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cb7a19aec52490a8d8229304f7776fb, State: Initialized, Role: FOLLOWER
I20260811 19:02:36.857461 27263 consensus_queue.cc:260] T 5363877ee27a42709bdf1c5e8f3ce39b P 5cb7a19aec52490a8d8229304f7776fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9382c1e24e04478c834ce2433b5ec413" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 36761 } } peers { permanent_uuid: "f4489642373b40788f9991efef9212bd" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 37705 } } peers { permanent_uuid: "5cb7a19aec52490a8d8229304f7776fb" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 46519 } }
I20260811 19:02:36.858268 27260 ts_tablet_manager.cc:1428] T 5363877ee27a42709bdf1c5e8f3ce39b P f4489642373b40788f9991efef9212bd: Time spent starting tablet: real 0.016s user 0.009s sys 0.009s
I20260811 19:02:36.860232 26981 heartbeater.cc:503] Master 127.29.112.190:46783 was elected leader, sending a full tablet report...
I20260811 19:02:36.861565 27263 ts_tablet_manager.cc:1428] T 5363877ee27a42709bdf1c5e8f3ce39b P 5cb7a19aec52490a8d8229304f7776fb: Time spent starting tablet: real 0.017s user 0.012s sys 0.003s
I20260811 19:02:36.883235 27273 raft_consensus.cc:482] T 5363877ee27a42709bdf1c5e8f3ce39b P 9382c1e24e04478c834ce2433b5ec413 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:02:36.883580 27273 raft_consensus.cc:504] T 5363877ee27a42709bdf1c5e8f3ce39b P 9382c1e24e04478c834ce2433b5ec413 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9382c1e24e04478c834ce2433b5ec413" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 36761 } } peers { permanent_uuid: "f4489642373b40788f9991efef9212bd" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 37705 } } peers { permanent_uuid: "5cb7a19aec52490a8d8229304f7776fb" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 46519 } }
I20260811 19:02:36.885648 27273 leader_election.cc:288] T 5363877ee27a42709bdf1c5e8f3ce39b P 9382c1e24e04478c834ce2433b5ec413 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f4489642373b40788f9991efef9212bd (127.29.112.129:37705), 5cb7a19aec52490a8d8229304f7776fb (127.29.112.130:46519)
I20260811 19:02:36.892205 27225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:36.885615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:37705 (local address 127.29.112.131:48677)
0811 19:02:36.885922 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:36.885944 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 19:02:36.886086 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:02:36.886517 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:02:36.886522 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:02:36.886629 (+ 107us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:02:36.886984 (+ 355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:36.887005 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:36.888091 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:36.888099 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:36.889717 (+ 1618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:36.889739 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:36.891233 (+ 1494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:36.891241 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:36.891544 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:36.891581 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 19:02:36.891722 (+ 141us) client_negotiation.cc:240] Negotiation successful
0811 19:02:36.891885 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147}
I20260811 19:02:36.892273 27254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:36.885879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.131:48677 (local address 127.29.112.129:37705)
0811 19:02:36.886122 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:36.886129 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:36.886153 (+ 24us) server_negotiation.cc:407] Connection header received
0811 19:02:36.886240 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:36.886248 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:36.886338 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:36.886467 (+ 129us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:02:36.887162 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:36.887942 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:36.889900 (+ 1958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:36.890887 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:36.891071 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:36.891156 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:36.891786 (+ 630us) server_negotiation.cc:299] Negotiation successful
0811 19:02:36.891956 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 19:02:36.893375 27258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:36.886134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.131:33381 (local address 127.29.112.130:46519)
0811 19:02:36.886328 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:36.886332 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:36.887263 (+ 931us) server_negotiation.cc:407] Connection header received
0811 19:02:36.887446 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:36.887450 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:36.887508 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:36.887615 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:02:36.888215 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:36.888992 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:36.890927 (+ 1935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:36.892044 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:36.892208 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:36.892292 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:36.892849 (+ 557us) server_negotiation.cc:299] Negotiation successful
0811 19:02:36.893041 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 19:02:36.893445 27276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:36.886167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.130:46519 (local address 127.29.112.131:33381)
0811 19:02:36.887149 (+ 982us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:36.887166 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 19:02:36.887308 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:02:36.887627 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:02:36.887632 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:02:36.887656 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:02:36.887991 (+ 335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:36.888023 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:36.889158 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:36.889164 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:36.890762 (+ 1598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:36.890774 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:36.892321 (+ 1547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:36.892328 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:36.892541 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:36.892575 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 19:02:36.892818 (+ 243us) client_negotiation.cc:240] Negotiation successful
0811 19:02:36.892990 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":832,"mutex_wait_us":4,"thread_start_us":581,"threads_started":1}
I20260811 19:02:36.893754 26681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5363877ee27a42709bdf1c5e8f3ce39b" candidate_uuid: "9382c1e24e04478c834ce2433b5ec413" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4489642373b40788f9991efef9212bd" is_pre_election: true
I20260811 19:02:36.894856 26900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5363877ee27a42709bdf1c5e8f3ce39b" candidate_uuid: "9382c1e24e04478c834ce2433b5ec413" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cb7a19aec52490a8d8229304f7776fb" is_pre_election: true
I20260811 19:02:36.894951 26681 raft_consensus.cc:2454] T 5363877ee27a42709bdf1c5e8f3ce39b P f4489642373b40788f9991efef9212bd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9382c1e24e04478c834ce2433b5ec413 in term 0.
I20260811 19:02:36.895435 26900 raft_consensus.cc:2454] T 5363877ee27a42709bdf1c5e8f3ce39b P 5cb7a19aec52490a8d8229304f7776fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9382c1e24e04478c834ce2433b5ec413 in term 0.
I20260811 19:02:36.896026 27028 leader_election.cc:302] T 5363877ee27a42709bdf1c5e8f3ce39b P 9382c1e24e04478c834ce2433b5ec413 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9382c1e24e04478c834ce2433b5ec413, f4489642373b40788f9991efef9212bd; no voters:
I20260811 19:02:36.896536 27273 raft_consensus.cc:2781] T 5363877ee27a42709bdf1c5e8f3ce39b P 9382c1e24e04478c834ce2433b5ec413 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 19:02:36.896688 27273 raft_consensus.cc:482] T 5363877ee27a42709bdf1c5e8f3ce39b P 9382c1e24e04478c834ce2433b5ec413 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 19:02:36.896807 27273 raft_consensus.cc:3037] T 5363877ee27a42709bdf1c5e8f3ce39b P 9382c1e24e04478c834ce2433b5ec413 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:02:36.899173 27273 raft_consensus.cc:504] T 5363877ee27a42709bdf1c5e8f3ce39b P 9382c1e24e04478c834ce2433b5ec413 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9382c1e24e04478c834ce2433b5ec413" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 36761 } } peers { permanent_uuid: "f4489642373b40788f9991efef9212bd" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 37705 } } peers { permanent_uuid: "5cb7a19aec52490a8d8229304f7776fb" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 46519 } }
I20260811 19:02:36.899775 27273 leader_election.cc:288] T 5363877ee27a42709bdf1c5e8f3ce39b P 9382c1e24e04478c834ce2433b5ec413 [CANDIDATE]: Term 1 election: Requested vote from peers f4489642373b40788f9991efef9212bd (127.29.112.129:37705), 5cb7a19aec52490a8d8229304f7776fb (127.29.112.130:46519)
I20260811 19:02:36.900388 26900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5363877ee27a42709bdf1c5e8f3ce39b" candidate_uuid: "9382c1e24e04478c834ce2433b5ec413" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cb7a19aec52490a8d8229304f7776fb"
I20260811 19:02:36.900389 26681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5363877ee27a42709bdf1c5e8f3ce39b" candidate_uuid: "9382c1e24e04478c834ce2433b5ec413" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4489642373b40788f9991efef9212bd"
I20260811 19:02:36.900605 26900 raft_consensus.cc:3037] T 5363877ee27a42709bdf1c5e8f3ce39b P 5cb7a19aec52490a8d8229304f7776fb [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:02:36.900609 26681 raft_consensus.cc:3037] T 5363877ee27a42709bdf1c5e8f3ce39b P f4489642373b40788f9991efef9212bd [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:02:36.903980 26681 raft_consensus.cc:2454] T 5363877ee27a42709bdf1c5e8f3ce39b P f4489642373b40788f9991efef9212bd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9382c1e24e04478c834ce2433b5ec413 in term 1.
I20260811 19:02:36.903980 26900 raft_consensus.cc:2454] T 5363877ee27a42709bdf1c5e8f3ce39b P 5cb7a19aec52490a8d8229304f7776fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9382c1e24e04478c834ce2433b5ec413 in term 1.
W20260811 19:02:36.904211 26755 tablet.cc:2369] T 5363877ee27a42709bdf1c5e8f3ce39b P f4489642373b40788f9991efef9212bd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 19:02:36.904474 27026 leader_election.cc:302] T 5363877ee27a42709bdf1c5e8f3ce39b P 9382c1e24e04478c834ce2433b5ec413 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cb7a19aec52490a8d8229304f7776fb, 9382c1e24e04478c834ce2433b5ec413; no voters:
I20260811 19:02:36.904831 27273 raft_consensus.cc:2781] T 5363877ee27a42709bdf1c5e8f3ce39b P 9382c1e24e04478c834ce2433b5ec413 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:02:36.905422 27273 raft_consensus.cc:686] T 5363877ee27a42709bdf1c5e8f3ce39b P 9382c1e24e04478c834ce2433b5ec413 [term 1 LEADER]: Becoming Leader. State: Replica: 9382c1e24e04478c834ce2433b5ec413, State: Running, Role: LEADER
I20260811 19:02:36.906000 27273 consensus_queue.cc:237] T 5363877ee27a42709bdf1c5e8f3ce39b P 9382c1e24e04478c834ce2433b5ec413 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9382c1e24e04478c834ce2433b5ec413" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 36761 } } peers { permanent_uuid: "f4489642373b40788f9991efef9212bd" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 37705 } } peers { permanent_uuid: "5cb7a19aec52490a8d8229304f7776fb" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 46519 } }
I20260811 19:02:36.910905 26453 catalog_manager.cc:5495] T 5363877ee27a42709bdf1c5e8f3ce39b P 9382c1e24e04478c834ce2433b5ec413 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9382c1e24e04478c834ce2433b5ec413 (127.29.112.131). New cstate: current_term: 1 leader_uuid: "9382c1e24e04478c834ce2433b5ec413" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9382c1e24e04478c834ce2433b5ec413" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 36761 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f4489642373b40788f9991efef9212bd" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 37705 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cb7a19aec52490a8d8229304f7776fb" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 46519 } health_report { overall_health: UNKNOWN } } }
I20260811 19:02:36.935735 27247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:36.928898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41362 (local address 127.29.112.131:36761)
0811 19:02:36.929085 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:36.929090 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:36.929481 (+ 391us) server_negotiation.cc:407] Connection header received
0811 19:02:36.929606 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:36.929610 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:36.929692 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:36.929835 (+ 143us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:02:36.930429 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:36.931249 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:36.932900 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:36.933422 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:36.933556 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:36.934259 (+ 703us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:02:36.935039 (+ 780us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:02:36.935194 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:36.935375 (+ 181us) server_negotiation.cc:299] Negotiation successful
0811 19:02:36.935478 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 19:02:36.944859 26900 raft_consensus.cc:1261] T 5363877ee27a42709bdf1c5e8f3ce39b P 5cb7a19aec52490a8d8229304f7776fb [term 1 FOLLOWER]: Refusing update from remote peer 9382c1e24e04478c834ce2433b5ec413: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:02:36.944864 26681 raft_consensus.cc:1261] T 5363877ee27a42709bdf1c5e8f3ce39b P f4489642373b40788f9991efef9212bd [term 1 FOLLOWER]: Refusing update from remote peer 9382c1e24e04478c834ce2433b5ec413: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260811 19:02:36.945261 27222 tablet.cc:2369] T 5363877ee27a42709bdf1c5e8f3ce39b P 9382c1e24e04478c834ce2433b5ec413: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 19:02:36.946134 27280 consensus_queue.cc:1035] T 5363877ee27a42709bdf1c5e8f3ce39b P 9382c1e24e04478c834ce2433b5ec413 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f4489642373b40788f9991efef9212bd" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 37705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:02:36.946710 27273 consensus_queue.cc:1035] T 5363877ee27a42709bdf1c5e8f3ce39b P 9382c1e24e04478c834ce2433b5ec413 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cb7a19aec52490a8d8229304f7776fb" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:02:36.957908 27283 mvcc.cc:205] Tried to move back new op lower bound from 7317401423632879616 to 7317401423491489792. Current Snapshot: MvccSnapshot[applied={T|T < 7317401423632879616}]
I20260811 19:02:36.959065 27286 mvcc.cc:205] Tried to move back new op lower bound from 7317401423632879616 to 7317401423491489792. Current Snapshot: MvccSnapshot[applied={T|T < 7317401423632879616}]
I20260811 19:02:36.973384 26453 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55374:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:02:36.973937 26453 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55374:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:02:36.975687 27284 mvcc.cc:205] Tried to move back new op lower bound from 7317401423632879616 to 7317401423491489792. Current Snapshot: MvccSnapshot[applied={T|T < 7317401423632879616}]
I20260811 19:02:36.981320 26453 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7708e71da9954be89003ba3ad26de8fe: Sending DeleteTablet for 3 replicas of tablet 5363877ee27a42709bdf1c5e8f3ce39b
I20260811 19:02:36.982589 26656 tablet_service.cc:1514] Processing DeleteTablet for tablet 5363877ee27a42709bdf1c5e8f3ce39b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:02:36 UTC) from {username='kudu', principal='kudu/127.29.112.190@KRBTEST.COM'} at 127.0.0.1:40948
I20260811 19:02:36.982906 27108 tablet_service.cc:1514] Processing DeleteTablet for tablet 5363877ee27a42709bdf1c5e8f3ce39b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:02:36 UTC) from {username='kudu', principal='kudu/127.29.112.190@KRBTEST.COM'} at 127.0.0.1:41356
I20260811 19:02:36.983116 26870 tablet_service.cc:1514] Processing DeleteTablet for tablet 5363877ee27a42709bdf1c5e8f3ce39b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:02:36 UTC) from {username='kudu', principal='kudu/127.29.112.190@KRBTEST.COM'} at 127.0.0.1:56990
I20260811 19:02:36.984174 27300 tablet_replica.cc:330] T 5363877ee27a42709bdf1c5e8f3ce39b P 5cb7a19aec52490a8d8229304f7776fb: stopping tablet replica
I20260811 19:02:36.984727 27300 raft_consensus.cc:2229] T 5363877ee27a42709bdf1c5e8f3ce39b P 5cb7a19aec52490a8d8229304f7776fb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:02:36.985392 27300 raft_consensus.cc:2258] T 5363877ee27a42709bdf1c5e8f3ce39b P 5cb7a19aec52490a8d8229304f7776fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:02:36.986501 26868 tablet_service.cc:1514] Processing DeleteTablet for tablet 5363877ee27a42709bdf1c5e8f3ce39b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:02:36 UTC) from {username='kudu', principal='kudu/127.29.112.190@KRBTEST.COM'} at 127.0.0.1:56990
I20260811 19:02:36.990749 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 26546
I20260811 19:02:36.995776 27300 ts_tablet_manager.cc:1905] T 5363877ee27a42709bdf1c5e8f3ce39b P 5cb7a19aec52490a8d8229304f7776fb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:02:37.003484 27298 tablet_replica.cc:330] T 5363877ee27a42709bdf1c5e8f3ce39b P f4489642373b40788f9991efef9212bd: stopping tablet replica
I20260811 19:02:37.003468 27299 tablet_replica.cc:330] T 5363877ee27a42709bdf1c5e8f3ce39b P 9382c1e24e04478c834ce2433b5ec413: stopping tablet replica
I20260811 19:02:37.004269 27298 raft_consensus.cc:2229] T 5363877ee27a42709bdf1c5e8f3ce39b P f4489642373b40788f9991efef9212bd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:02:37.004617 27299 raft_consensus.cc:2229] T 5363877ee27a42709bdf1c5e8f3ce39b P 9382c1e24e04478c834ce2433b5ec413 [term 1 LEADER]: Raft consensus shutting down.
I20260811 19:02:37.004959 27298 raft_consensus.cc:2258] T 5363877ee27a42709bdf1c5e8f3ce39b P f4489642373b40788f9991efef9212bd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:02:37.005843 27299 raft_consensus.cc:2258] T 5363877ee27a42709bdf1c5e8f3ce39b P 9382c1e24e04478c834ce2433b5ec413 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:02:37.008078 27300 ts_tablet_manager.cc:1918] T 5363877ee27a42709bdf1c5e8f3ce39b P 5cb7a19aec52490a8d8229304f7776fb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:02:37.008311 27298 ts_tablet_manager.cc:1905] T 5363877ee27a42709bdf1c5e8f3ce39b P f4489642373b40788f9991efef9212bd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:02:37.008313 27300 log.cc:1196] T 5363877ee27a42709bdf1c5e8f3ce39b P 5cb7a19aec52490a8d8229304f7776fb: Deleting WAL directory at /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/minicluster-data/ts-1/wal/wals/5363877ee27a42709bdf1c5e8f3ce39b
I20260811 19:02:37.009017 27300 ts_tablet_manager.cc:1939] T 5363877ee27a42709bdf1c5e8f3ce39b P 5cb7a19aec52490a8d8229304f7776fb: Deleting consensus metadata
I20260811 19:02:37.011255 26423 catalog_manager.cc:4841] TS 5cb7a19aec52490a8d8229304f7776fb (127.29.112.130:46519): tablet 5363877ee27a42709bdf1c5e8f3ce39b (table test-table [id=6a83f591f58e41dfb415f1902dbdd3fa]) successfully deleted
W20260811 19:02:37.012004 26423 catalog_manager.cc:4808] TS 5cb7a19aec52490a8d8229304f7776fb (127.29.112.130:46519): delete failed for tablet 5363877ee27a42709bdf1c5e8f3ce39b because the tablet was not found. No further retry: Not found: Tablet not found: 5363877ee27a42709bdf1c5e8f3ce39b
I20260811 19:02:37.013777 27299 ts_tablet_manager.cc:1905] T 5363877ee27a42709bdf1c5e8f3ce39b P 9382c1e24e04478c834ce2433b5ec413: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:02:37.016968 27298 ts_tablet_manager.cc:1918] T 5363877ee27a42709bdf1c5e8f3ce39b P f4489642373b40788f9991efef9212bd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:02:37.017155 27298 log.cc:1196] T 5363877ee27a42709bdf1c5e8f3ce39b P f4489642373b40788f9991efef9212bd: Deleting WAL directory at /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/minicluster-data/ts-0/wal/wals/5363877ee27a42709bdf1c5e8f3ce39b
I20260811 19:02:37.017688 27298 ts_tablet_manager.cc:1939] T 5363877ee27a42709bdf1c5e8f3ce39b P f4489642373b40788f9991efef9212bd: Deleting consensus metadata
I20260811 19:02:37.019363 26427 catalog_manager.cc:4841] TS f4489642373b40788f9991efef9212bd (127.29.112.129:37705): tablet 5363877ee27a42709bdf1c5e8f3ce39b (table test-table [id=6a83f591f58e41dfb415f1902dbdd3fa]) successfully deleted
Aug 11 19:02:37 dist-test-slave-h6n0 krb5kdc[26367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474953, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.129@KRBTEST.COM
I20260811 19:02:37.026721 27299 ts_tablet_manager.cc:1918] T 5363877ee27a42709bdf1c5e8f3ce39b P 9382c1e24e04478c834ce2433b5ec413: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:02:37.026988 27299 log.cc:1196] T 5363877ee27a42709bdf1c5e8f3ce39b P 9382c1e24e04478c834ce2433b5ec413: Deleting WAL directory at /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474615205622-30146-0/minicluster-data/ts-2/wal/wals/5363877ee27a42709bdf1c5e8f3ce39b
I20260811 19:02:37.027657 27299 ts_tablet_manager.cc:1939] T 5363877ee27a42709bdf1c5e8f3ce39b P 9382c1e24e04478c834ce2433b5ec413: Deleting consensus metadata
I20260811 19:02:37.029945 26423 catalog_manager.cc:4841] TS 9382c1e24e04478c834ce2433b5ec413 (127.29.112.131:36761): tablet 5363877ee27a42709bdf1c5e8f3ce39b (table test-table [id=6a83f591f58e41dfb415f1902dbdd3fa]) successfully deleted
I20260811 19:02:37.032394 27254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:36.992053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40956 (local address 127.29.112.129:37705)
0811 19:02:36.992251 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:36.992258 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:37.005864 (+ 13606us) server_negotiation.cc:407] Connection header received
0811 19:02:37.007093 (+ 1229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:37.007098 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:37.007194 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:37.007280 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:02:37.009363 (+ 2083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:37.010606 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:37.012692 (+ 2086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:37.013385 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:37.013508 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:37.023956 (+ 10448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:02:37.023973 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:02:37.023989 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:02:37.024037 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:02:37.027978 (+ 3941us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:37.029093 (+ 1115us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:37.029103 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:37.029106 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:37.029433 (+ 327us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:37.030284 (+ 851us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:37.030292 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:37.030295 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:37.030991 (+ 696us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:02:37.031422 (+ 431us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:37.031859 (+ 437us) server_negotiation.cc:299] Negotiation successful
0811 19:02:37.032089 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 19:02:37.033762 26742 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:02:37.135824 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 26546
I20260811 19:02:37.172793 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 26766
Aug 11 19:02:37 dist-test-slave-h6n0 krb5kdc[26367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474953, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.130@KRBTEST.COM
I20260811 19:02:37.197970 27258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:37.174218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56998 (local address 127.29.112.130:46519)
0811 19:02:37.174818 (+ 600us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:37.174827 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:37.174856 (+ 29us) server_negotiation.cc:407] Connection header received
0811 19:02:37.174959 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:37.174966 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:37.175090 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:37.175441 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:02:37.177083 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:37.178720 (+ 1637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:37.180955 (+ 2235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:37.181775 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:37.181954 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:37.187561 (+ 5607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:02:37.187584 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:02:37.187607 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:02:37.187662 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:02:37.192641 (+ 4979us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:37.194026 (+ 1385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:37.194039 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:37.194044 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:37.194296 (+ 252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:37.195139 (+ 843us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:37.195146 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:37.195152 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:37.196213 (+ 1061us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:02:37.196795 (+ 582us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:37.197035 (+ 240us) server_negotiation.cc:299] Negotiation successful
0811 19:02:37.197493 (+ 458us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":255}
I20260811 19:02:37.199332 26973 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:02:37.302237 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 26766
I20260811 19:02:37.348867 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 26994
Aug 11 19:02:37 dist-test-slave-h6n0 krb5kdc[26367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474953, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.131@KRBTEST.COM
I20260811 19:02:37.370409 27247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:37.350310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41366 (local address 127.29.112.131:36761)
0811 19:02:37.350492 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:37.350497 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:37.350512 (+ 15us) server_negotiation.cc:407] Connection header received
0811 19:02:37.350619 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:37.350623 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:37.350715 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:37.350843 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:02:37.352212 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:37.353058 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:37.354406 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:37.355038 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:37.355190 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:37.361707 (+ 6517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:02:37.361720 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:02:37.361736 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:02:37.361782 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:02:37.365769 (+ 3987us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:37.367446 (+ 1677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:37.367461 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:37.367463 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:37.367731 (+ 268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:37.368191 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:37.368198 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:37.368202 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:37.368750 (+ 548us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:02:37.369291 (+ 541us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:37.369738 (+ 447us) server_negotiation.cc:299] Negotiation successful
0811 19:02:37.370017 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 19:02:37.371665 27215 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:02:37.506785 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 26994
I20260811 19:02:37.536607 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 26392
I20260811 19:02:37.538930 26503 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:02:37.668643 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 26392
2026-08-11T19:02:37Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (5470 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-11T19:02:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:02:37Z Disabled control of system clock
I20260811 19:02:37.786170 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474615205622-30146-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474615205622-30146-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474615205622-30146-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.190:42977
--webserver_interface=127.29.112.190
--webserver_port=0
--builtin_ntp_servers=127.29.112.148:37747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.190:42977
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 19:02:38.185948 27318 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:02:38.186733 27318 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:02:38.186867 27318 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:02:38.195700 27318 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:02:38.195851 27318 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:02:38.195910 27318 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:02:38.195964 27318 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:02:38.196012 27318 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:02:38.209090 27318 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:37747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474615205622-30146-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474615205622-30146-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.112.190:42977
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.190:42977
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.112.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474615205622-30146-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:02:38.211374 27318 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:02:38.213577 27318 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:02:38.232615 27327 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:02:38.232656 27328 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:02:38.232597 27330 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:02:38.235088 27318 server_base.cc:1034] running on GCE node
I20260811 19:02:38.237114 27318 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:02:38.240298 27318 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:02:38.242374 27318 hybrid_clock.cc:648] HybridClock initialized: now 1786474958241964 us; error 423 us; skew 500 ppm
I20260811 19:02:38.243029 27318 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 19:02:38.249289 27318 webserver.cc:460] Webserver started at http://127.29.112.190:39987/ using document root <none> and password file <none>
I20260811 19:02:38.250494 27318 fs_manager.cc:359] Metadata directory not provided
I20260811 19:02:38.250756 27318 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:02:38.251307 27318 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:02:38.255244 27318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474615205622-30146-0/minicluster-data/master-0/data/instance:
uuid: "3b6045caba42449daf034232f450bbe6"
format_stamp: "Formatted at 2026-08-11 19:02:38 on dist-test-slave-h6n0"
server_key: "e6489e77554266035ca1c2e5deb7bee2"
server_key_iv: "e64551b514f0fa0501e5327f64e60f5f"
server_key_version: "encryptionkey@0"
I20260811 19:02:38.256222 27318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474615205622-30146-0/minicluster-data/master-0/wal/instance:
uuid: "3b6045caba42449daf034232f450bbe6"
format_stamp: "Formatted at 2026-08-11 19:02:38 on dist-test-slave-h6n0"
server_key: "e6489e77554266035ca1c2e5deb7bee2"
server_key_iv: "e64551b514f0fa0501e5327f64e60f5f"
server_key_version: "encryptionkey@0"
I20260811 19:02:38.263465 27318 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.002s sys 0.005s
I20260811 19:02:38.268903 27343 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:38.272364 27318 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.006s sys 0.000s
I20260811 19:02:38.272917 27318 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474615205622-30146-0/minicluster-data/master-0/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474615205622-30146-0/minicluster-data/master-0/wal
uuid: "3b6045caba42449daf034232f450bbe6"
format_stamp: "Formatted at 2026-08-11 19:02:38 on dist-test-slave-h6n0"
server_key: "e6489e77554266035ca1c2e5deb7bee2"
server_key_iv: "e64551b514f0fa0501e5327f64e60f5f"
server_key_version: "encryptionkey@0"
I20260811 19:02:38.273541 27318 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474615205622-30146-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474615205622-30146-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474615205622-30146-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:02:38.305917 27318 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:02:38.307006 27318 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:02:38.307358 27318 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:02:38.334858 27318 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.190:42977
I20260811 19:02:38.334887 27443 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.190:42977 every 8 connection(s)
I20260811 19:02:38.336923 27318 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
I20260811 19:02:38.342185 27444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:02:38.346859 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 27318
I20260811 19:02:38.347121 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474615205622-30146-0/minicluster-data/master-0/wal/instance
I20260811 19:02:38.347856 30146 external_mini_cluster.cc:1428] Setting key cc62b45d7f684c29768be8cff49d94c8
I20260811 19:02:38.357801 27444 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b6045caba42449daf034232f450bbe6: Bootstrap starting.
I20260811 19:02:38.361395 27444 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3b6045caba42449daf034232f450bbe6: Neither blocks nor log segments found. Creating new log.
I20260811 19:02:38.363076 27444 log.cc:824] T 00000000000000000000000000000000 P 3b6045caba42449daf034232f450bbe6: Log is configured to *not* fsync() on all Append() calls
I20260811 19:02:38.366477 27450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:38.350451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49146 (local address 127.29.112.190:42977)
0811 19:02:38.351983 (+ 1532us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:38.351997 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:38.352049 (+ 52us) server_negotiation.cc:407] Connection header received
0811 19:02:38.354031 (+ 1982us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:38.354058 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:38.354751 (+ 693us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:38.355285 (+ 534us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:02:38.355999 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:38.357445 (+ 1446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:38.359275 (+ 1830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:38.360057 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:38.360284 (+ 227us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:38.361917 (+ 1633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:02:38.361925 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:02:38.362081 (+ 156us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:02:38.362994 (+ 913us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:02:38.363116 (+ 122us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:02:38.363141 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:02:38.363318 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:38.364045 (+ 727us) server_negotiation.cc:299] Negotiation successful
0811 19:02:38.364467 (+ 422us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":842,"thread_start_us":523,"threads_started":1}
I20260811 19:02:38.368813 27444 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b6045caba42449daf034232f450bbe6: No bootstrap required, opened a new log
I20260811 19:02:38.378392 27444 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3b6045caba42449daf034232f450bbe6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b6045caba42449daf034232f450bbe6" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 42977 } }
I20260811 19:02:38.379001 27444 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3b6045caba42449daf034232f450bbe6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:02:38.379150 27444 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3b6045caba42449daf034232f450bbe6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b6045caba42449daf034232f450bbe6, State: Initialized, Role: FOLLOWER
I20260811 19:02:38.379925 27444 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3b6045caba42449daf034232f450bbe6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b6045caba42449daf034232f450bbe6" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 42977 } }
I20260811 19:02:38.380210 27444 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3b6045caba42449daf034232f450bbe6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:02:38.380327 27444 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3b6045caba42449daf034232f450bbe6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:02:38.380463 27444 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3b6045caba42449daf034232f450bbe6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:02:38.383126 27444 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3b6045caba42449daf034232f450bbe6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b6045caba42449daf034232f450bbe6" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 42977 } }
I20260811 19:02:38.384318 27444 leader_election.cc:302] T 00000000000000000000000000000000 P 3b6045caba42449daf034232f450bbe6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3b6045caba42449daf034232f450bbe6; no voters:
I20260811 19:02:38.385528 27444 leader_election.cc:288] T 00000000000000000000000000000000 P 3b6045caba42449daf034232f450bbe6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:02:38.385718 27455 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3b6045caba42449daf034232f450bbe6 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:02:38.387383 27455 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3b6045caba42449daf034232f450bbe6 [term 1 LEADER]: Becoming Leader. State: Replica: 3b6045caba42449daf034232f450bbe6, State: Running, Role: LEADER
I20260811 19:02:38.387982 27455 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3b6045caba42449daf034232f450bbe6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b6045caba42449daf034232f450bbe6" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 42977 } }
I20260811 19:02:38.389063 27444 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3b6045caba42449daf034232f450bbe6 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:02:38.391270 27456 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b6045caba42449daf034232f450bbe6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3b6045caba42449daf034232f450bbe6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b6045caba42449daf034232f450bbe6" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 42977 } } }
I20260811 19:02:38.391172 27459 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b6045caba42449daf034232f450bbe6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3b6045caba42449daf034232f450bbe6. Latest consensus state: current_term: 1 leader_uuid: "3b6045caba42449daf034232f450bbe6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b6045caba42449daf034232f450bbe6" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 42977 } } }
I20260811 19:02:38.391918 27456 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b6045caba42449daf034232f450bbe6 [sys.catalog]: This master's current role is: LEADER
I20260811 19:02:38.391993 27459 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b6045caba42449daf034232f450bbe6 [sys.catalog]: This master's current role is: LEADER
I20260811 19:02:38.398492 27462 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:02:38.407485 27462 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:02:38.421010 27462 catalog_manager.cc:1347] Generated new cluster ID: c3bfd361993f40edb51015d7ca1d5b77
I20260811 19:02:38.421150 27462 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:02:38.431771 27462 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:02:38.433883 27462 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:02:38.441028 27462 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3b6045caba42449daf034232f450bbe6: Generated new TSK 0
I20260811 19:02:38.442169 27462 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 19:02:38.455065 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474615205622-30146-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474615205622-30146-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474615205622-30146-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474615205622-30146-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.129:0
--local_ip_for_outbound_sockets=127.29.112.129
--webserver_interface=127.29.112.129
--webserver_port=0
--tserver_master_addrs=127.29.112.190:42977
--builtin_ntp_servers=127.29.112.148:37747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 19:02:38.873973 27490 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:02:38.874281 27490 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:02:38.874383 27490 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:02:38.881894 27490 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:02:38.882011 27490 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:02:38.882161 27490 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.129
I20260811 19:02:38.892689 27490 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:37747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474615205622-30146-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474615205622-30146-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474615205622-30146-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.112.129
--webserver_port=0
--tserver_master_addrs=127.29.112.190:42977
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.129
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474615205622-30146-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:02:38.894104 27490 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:02:38.897050 27490 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:02:38.914279 27497 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:02:38.914256 27501 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:02:38.914317 27496 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:02:38.915802 27490 server_base.cc:1034] running on GCE node
I20260811 19:02:38.916633 27490 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:02:38.917989 27490 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:02:38.919452 27490 hybrid_clock.cc:648] HybridClock initialized: now 1786474958919359 us; error 71 us; skew 500 ppm
I20260811 19:02:38.919869 27490 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 19:02:38.923063 27490 webserver.cc:460] Webserver started at http://127.29.112.129:40123/ using document root <none> and password file <none>
I20260811 19:02:38.923815 27490 fs_manager.cc:359] Metadata directory not provided
I20260811 19:02:38.923957 27490 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:02:38.924299 27490 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:02:38.927485 27490 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474615205622-30146-0/minicluster-data/ts-0/data/instance:
uuid: "9b7d6cb727c94e7cb9512e344001b0eb"
format_stamp: "Formatted at 2026-08-11 19:02:38 on dist-test-slave-h6n0"
server_key: "0034152a08cb6f24e65aef979ec5467e"
server_key_iv: "5cf1ef445569840444caa1067a84ae00"
server_key_version: "encryptionkey@0"
I20260811 19:02:38.928403 27490 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474615205622-30146-0/minicluster-data/ts-0/wal/instance:
uuid: "9b7d6cb727c94e7cb9512e344001b0eb"
format_stamp: "Formatted at 2026-08-11 19:02:38 on dist-test-slave-h6n0"
server_key: "0034152a08cb6f24e65aef979ec5467e"
server_key_iv: "5cf1ef445569840444caa1067a84ae00"
server_key_version: "encryptionkey@0"
I20260811 19:02:38.933851 27490 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260811 19:02:38.937929 27513 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:38.940193 27490 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.003s
I20260811 19:02:38.940508 27490 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474615205622-30146-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474615205622-30146-0/minicluster-data/ts-0/wal
uuid: "9b7d6cb727c94e7cb9512e344001b0eb"
format_stamp: "Formatted at 2026-08-11 19:02:38 on dist-test-slave-h6n0"
server_key: "0034152a08cb6f24e65aef979ec5467e"
server_key_iv: "5cf1ef445569840444caa1067a84ae00"
server_key_version: "encryptionkey@0"
I20260811 19:02:38.941041 27490 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474615205622-30146-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474615205622-30146-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474615205622-30146-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:02:38.968179 27490 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:02:38.969097 27490 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:02:38.969427 27490 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:02:38.970477 27490 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:02:38.972116 27490 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:02:38.972249 27490 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:38.972393 27490 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:02:38.972487 27490 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:39.009707 27490 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.129:41329
I20260811 19:02:39.009819 27721 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.129:41329 every 8 connection(s)
I20260811 19:02:39.011112 27490 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474615205622-30146-0/minicluster-data/ts-0/data/info.pb
I20260811 19:02:39.014205 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 27490
I20260811 19:02:39.014578 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474615205622-30146-0/minicluster-data/ts-0/wal/instance
I20260811 19:02:39.015269 30146 external_mini_cluster.cc:1428] Setting key 2a1e3f0022e1450ecc70c5bdb4ef6c54
I20260811 19:02:39.018560 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474615205622-30146-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474615205622-30146-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474615205622-30146-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474615205622-30146-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.130:0
--local_ip_for_outbound_sockets=127.29.112.130
--webserver_interface=127.29.112.130
--webserver_port=0
--tserver_master_addrs=127.29.112.190:42977
--builtin_ntp_servers=127.29.112.148:37747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 19:02:39.027868 27728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:39.013937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.129:49145 (local address 127.29.112.190:42977)
0811 19:02:39.014948 (+ 1011us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:39.014955 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:39.017878 (+ 2923us) server_negotiation.cc:407] Connection header received
0811 19:02:39.017930 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:39.017934 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:39.018002 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:39.018087 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:02:39.019746 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:39.020687 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:39.023689 (+ 3002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:39.024491 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:39.024613 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:39.026778 (+ 2165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:02:39.026784 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:02:39.026787 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:02:39.026813 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:02:39.026867 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:02:39.026881 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:02:39.026972 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:39.027322 (+ 350us) server_negotiation.cc:299] Negotiation successful
0811 19:02:39.027425 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":782,"thread_start_us":459,"threads_started":1}
I20260811 19:02:39.029376 27729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:39.014595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:42977 (local address 127.29.112.129:49145)
0811 19:02:39.015660 (+ 1065us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:39.015716 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 19:02:39.017606 (+ 1890us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:02:39.018430 (+ 824us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:02:39.018443 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:02:39.018611 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:02:39.019504 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:39.019535 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:39.020872 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:39.020881 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:39.023483 (+ 2602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:39.023503 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:39.024774 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:39.024785 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:39.025170 (+ 385us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:39.026105 (+ 935us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 19:02:39.026111 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:02:39.026174 (+ 63us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 19:02:39.026302 (+ 128us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:02:39.027007 (+ 705us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:02:39.027023 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:02:39.027037 (+ 14us) client_negotiation.cc:769] Sending connection context
0811 19:02:39.027248 (+ 211us) client_negotiation.cc:240] Negotiation successful
0811 19:02:39.027781 (+ 533us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":366,"threads_started":1}
I20260811 19:02:39.032585 27722 heartbeater.cc:347] Connected to a master server at 127.29.112.190:42977
I20260811 19:02:39.033236 27722 heartbeater.cc:464] Registering TS with master...
I20260811 19:02:39.034629 27722 heartbeater.cc:511] Master 127.29.112.190:42977 requested a full tablet report, sending...
I20260811 19:02:39.037675 27370 ts_manager.cc:194] Registered new tserver with Master: 9b7d6cb727c94e7cb9512e344001b0eb (127.29.112.129:41329)
I20260811 19:02:39.040021 27370 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.112.129:49145
W20260811 19:02:39.388926 27730 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:02:39.389299 27730 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:02:39.389405 27730 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:02:39.396867 27730 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:02:39.396972 27730 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:02:39.397080 27730 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.130
I20260811 19:02:39.406877 27730 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:37747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474615205622-30146-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474615205622-30146-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474615205622-30146-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.112.130
--webserver_port=0
--tserver_master_addrs=127.29.112.190:42977
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.130
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474615205622-30146-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:02:39.408262 27730 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:02:39.410210 27730 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:02:39.426378 27736 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:02:39.426339 27740 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:02:39.426447 27737 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:02:39.428751 27730 server_base.cc:1034] running on GCE node
I20260811 19:02:39.429386 27730 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:02:39.430366 27730 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:02:39.431633 27730 hybrid_clock.cc:648] HybridClock initialized: now 1786474959431565 us; error 59 us; skew 500 ppm
I20260811 19:02:39.431941 27730 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 19:02:39.434337 27730 webserver.cc:460] Webserver started at http://127.29.112.130:40549/ using document root <none> and password file <none>
I20260811 19:02:39.435124 27730 fs_manager.cc:359] Metadata directory not provided
I20260811 19:02:39.435225 27730 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:02:39.435518 27730 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:02:39.438522 27730 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474615205622-30146-0/minicluster-data/ts-1/data/instance:
uuid: "bf87d2e4a0704ac9925ff8bd593fbfb4"
format_stamp: "Formatted at 2026-08-11 19:02:39 on dist-test-slave-h6n0"
server_key: "a3f1820f8fa2bb5db32d97c825b82afc"
server_key_iv: "06c51824dedcc0a473ecfc2d93f33525"
server_key_version: "encryptionkey@0"
I20260811 19:02:39.439425 27730 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474615205622-30146-0/minicluster-data/ts-1/wal/instance:
uuid: "bf87d2e4a0704ac9925ff8bd593fbfb4"
format_stamp: "Formatted at 2026-08-11 19:02:39 on dist-test-slave-h6n0"
server_key: "a3f1820f8fa2bb5db32d97c825b82afc"
server_key_iv: "06c51824dedcc0a473ecfc2d93f33525"
server_key_version: "encryptionkey@0"
I20260811 19:02:39.446044 27730 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 19:02:39.451396 27751 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:39.453114 27730 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 19:02:39.453291 27730 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474615205622-30146-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474615205622-30146-0/minicluster-data/ts-1/wal
uuid: "bf87d2e4a0704ac9925ff8bd593fbfb4"
format_stamp: "Formatted at 2026-08-11 19:02:39 on dist-test-slave-h6n0"
server_key: "a3f1820f8fa2bb5db32d97c825b82afc"
server_key_iv: "06c51824dedcc0a473ecfc2d93f33525"
server_key_version: "encryptionkey@0"
I20260811 19:02:39.453516 27730 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474615205622-30146-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474615205622-30146-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474615205622-30146-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:02:39.474880 27730 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:02:39.475797 27730 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:02:39.476104 27730 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:02:39.477237 27730 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:02:39.478859 27730 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:02:39.478960 27730 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:39.479050 27730 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:02:39.479112 27730 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:39.523878 27730 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.130:40083
I20260811 19:02:39.523976 27951 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.130:40083 every 8 connection(s)
I20260811 19:02:39.525295 27730 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474615205622-30146-0/minicluster-data/ts-1/data/info.pb
I20260811 19:02:39.529711 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 27730
I20260811 19:02:39.529979 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474615205622-30146-0/minicluster-data/ts-1/wal/instance
I20260811 19:02:39.530625 30146 external_mini_cluster.cc:1428] Setting key 89dba825a58891779907bde20f9200d6
I20260811 19:02:39.534492 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474615205622-30146-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474615205622-30146-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474615205622-30146-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474615205622-30146-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.131:0
--local_ip_for_outbound_sockets=127.29.112.131
--webserver_interface=127.29.112.131
--webserver_port=0
--tserver_master_addrs=127.29.112.190:42977
--builtin_ntp_servers=127.29.112.148:37747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 19:02:39.540256 27728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:39.528214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.130:54499 (local address 127.29.112.190:42977)
0811 19:02:39.528445 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:39.528450 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:39.530660 (+ 2210us) server_negotiation.cc:407] Connection header received
0811 19:02:39.533593 (+ 2933us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:39.533598 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:39.533690 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:39.533819 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:02:39.535099 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:39.535821 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:39.537408 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:39.537799 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:39.537937 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:39.539424 (+ 1487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:02:39.539429 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:02:39.539432 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:02:39.539452 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:02:39.539483 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:02:39.539496 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:02:39.539598 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:39.539839 (+ 241us) server_negotiation.cc:299] Negotiation successful
0811 19:02:39.539937 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260811 19:02:39.541330 27954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:39.528705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:42977 (local address 127.29.112.130:54499)
0811 19:02:39.530272 (+ 1567us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:39.530369 (+ 97us) client_negotiation.cc:174] Beginning negotiation
0811 19:02:39.533242 (+ 2873us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:02:39.534063 (+ 821us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:02:39.534071 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:02:39.534166 (+ 95us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:02:39.534931 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:39.534952 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:39.535986 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:39.535992 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:39.537260 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:39.537272 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:39.538031 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:39.538039 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:39.538268 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:39.539082 (+ 814us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 19:02:39.539085 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:02:39.539126 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 19:02:39.539211 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:02:39.539598 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:02:39.539612 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:02:39.539621 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 19:02:39.539807 (+ 186us) client_negotiation.cc:240] Negotiation successful
0811 19:02:39.540229 (+ 422us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":717,"thread_start_us":449,"threads_started":1}
I20260811 19:02:39.543488 27952 heartbeater.cc:347] Connected to a master server at 127.29.112.190:42977
I20260811 19:02:39.544062 27952 heartbeater.cc:464] Registering TS with master...
I20260811 19:02:39.544947 27952 heartbeater.cc:511] Master 127.29.112.190:42977 requested a full tablet report, sending...
I20260811 19:02:39.546684 27370 ts_manager.cc:194] Registered new tserver with Master: bf87d2e4a0704ac9925ff8bd593fbfb4 (127.29.112.130:40083)
I20260811 19:02:39.547758 27370 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.112.130:54499
W20260811 19:02:39.931566 27956 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:02:39.931862 27956 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:02:39.932014 27956 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:02:39.939746 27956 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:02:39.939855 27956 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:02:39.940021 27956 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.131
I20260811 19:02:39.949978 27956 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:37747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474615205622-30146-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474615205622-30146-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474615205622-30146-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.112.131
--webserver_port=0
--tserver_master_addrs=127.29.112.190:42977
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.131
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474615205622-30146-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:02:39.951332 27956 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:02:39.953101 27956 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:02:39.968194 27963 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:02:39.968264 27968 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:02:39.968297 27964 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:02:39.969820 27956 server_base.cc:1034] running on GCE node
I20260811 19:02:39.970605 27956 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:02:39.971721 27956 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:02:39.973150 27956 hybrid_clock.cc:648] HybridClock initialized: now 1786474959973059 us; error 66 us; skew 500 ppm
I20260811 19:02:39.973564 27956 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 19:02:39.976423 27956 webserver.cc:460] Webserver started at http://127.29.112.131:42795/ using document root <none> and password file <none>
I20260811 19:02:39.977205 27956 fs_manager.cc:359] Metadata directory not provided
I20260811 19:02:39.977303 27956 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:02:39.977578 27956 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:02:39.980682 27956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474615205622-30146-0/minicluster-data/ts-2/data/instance:
uuid: "eaecfc7d38694ae294c422ef6ca7ca3d"
format_stamp: "Formatted at 2026-08-11 19:02:39 on dist-test-slave-h6n0"
server_key: "b167da2d253e9684fcb848f23571e154"
server_key_iv: "1c4032825c9f36ef15765ddeb6b9382b"
server_key_version: "encryptionkey@0"
I20260811 19:02:39.981703 27956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474615205622-30146-0/minicluster-data/ts-2/wal/instance:
uuid: "eaecfc7d38694ae294c422ef6ca7ca3d"
format_stamp: "Formatted at 2026-08-11 19:02:39 on dist-test-slave-h6n0"
server_key: "b167da2d253e9684fcb848f23571e154"
server_key_iv: "1c4032825c9f36ef15765ddeb6b9382b"
server_key_version: "encryptionkey@0"
I20260811 19:02:39.987349 27956 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 19:02:39.991472 27977 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:39.993011 27956 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 19:02:39.993177 27956 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474615205622-30146-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474615205622-30146-0/minicluster-data/ts-2/wal
uuid: "eaecfc7d38694ae294c422ef6ca7ca3d"
format_stamp: "Formatted at 2026-08-11 19:02:39 on dist-test-slave-h6n0"
server_key: "b167da2d253e9684fcb848f23571e154"
server_key_iv: "1c4032825c9f36ef15765ddeb6b9382b"
server_key_version: "encryptionkey@0"
I20260811 19:02:39.993441 27956 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474615205622-30146-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474615205622-30146-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474615205622-30146-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:02:40.006079 27956 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:02:40.007061 27956 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:02:40.007392 27956 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:02:40.008445 27956 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:02:40.010433 27956 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:02:40.010583 27956 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:40.010712 27956 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:02:40.010818 27956 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:40.044328 27722 heartbeater.cc:503] Master 127.29.112.190:42977 was elected leader, sending a full tablet report...
I20260811 19:02:40.049130 27956 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.131:39003
I20260811 19:02:40.049181 28183 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.131:39003 every 8 connection(s)
I20260811 19:02:40.050443 27956 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474615205622-30146-0/minicluster-data/ts-2/data/info.pb
I20260811 19:02:40.052851 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 27956
I20260811 19:02:40.053102 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474615205622-30146-0/minicluster-data/ts-2/wal/instance
I20260811 19:02:40.053682 30146 external_mini_cluster.cc:1428] Setting key 9b4df0070f14bcaed69262d81f5bcb7e
I20260811 19:02:40.065418 28189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:40.053350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.131:52003 (local address 127.29.112.190:42977)
0811 19:02:40.053859 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:40.053866 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:40.055257 (+ 1391us) server_negotiation.cc:407] Connection header received
0811 19:02:40.058019 (+ 2762us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:40.058025 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:40.058186 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:40.058461 (+ 275us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:02:40.059659 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:40.060409 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:40.062013 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:40.062434 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:40.062600 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:40.064409 (+ 1809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:02:40.064414 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:02:40.064417 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:02:40.064437 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:02:40.064480 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:02:40.064493 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:02:40.064607 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:40.064873 (+ 266us) server_negotiation.cc:299] Negotiation successful
0811 19:02:40.064985 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":252,"threads_started":1}
I20260811 19:02:40.067018 28191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:40.053871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:42977 (local address 127.29.112.131:52003)
0811 19:02:40.055110 (+ 1239us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:40.055150 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 19:02:40.057628 (+ 2478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:02:40.058749 (+ 1121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:02:40.058759 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:02:40.058865 (+ 106us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:02:40.059464 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:40.059483 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:40.060580 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:40.060584 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:40.061858 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:40.061872 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:40.062692 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:40.062701 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:40.062929 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:40.063741 (+ 812us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 19:02:40.063745 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:02:40.063816 (+ 71us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 19:02:40.063943 (+ 127us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:02:40.064609 (+ 666us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:02:40.064624 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:02:40.064634 (+ 10us) client_negotiation.cc:769] Sending connection context
0811 19:02:40.064837 (+ 203us) client_negotiation.cc:240] Negotiation successful
0811 19:02:40.065253 (+ 416us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":597,"thread_start_us":337,"threads_started":1}
I20260811 19:02:40.069250 28186 heartbeater.cc:347] Connected to a master server at 127.29.112.190:42977
I20260811 19:02:40.069654 28186 heartbeater.cc:464] Registering TS with master...
I20260811 19:02:40.070454 28186 heartbeater.cc:511] Master 127.29.112.190:42977 requested a full tablet report, sending...
I20260811 19:02:40.072170 27370 ts_manager.cc:194] Registered new tserver with Master: eaecfc7d38694ae294c422ef6ca7ca3d (127.29.112.131:39003)
I20260811 19:02:40.073367 27370 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.112.131:52003
I20260811 19:02:40.084323 30146 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 19:02:40.098703 28189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:40.089007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39460 (local address 127.29.112.190:42977)
0811 19:02:40.089237 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:40.089243 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:40.090335 (+ 1092us) server_negotiation.cc:407] Connection header received
0811 19:02:40.090634 (+ 299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:40.090640 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:40.090732 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:40.090887 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:02:40.091540 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:40.092361 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:40.094052 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:40.094996 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:40.095292 (+ 296us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:40.097584 (+ 2292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:02:40.097606 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:02:40.097617 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:02:40.097672 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:02:40.097741 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:02:40.097767 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:02:40.097911 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:40.098125 (+ 214us) server_negotiation.cc:299] Negotiation successful
0811 19:02:40.098270 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 19:02:40.104537 27370 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39460:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 19:02:40.112417 27370 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 19:02:40.142458 28214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:40.125951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59890 (local address 127.29.112.131:39003)
0811 19:02:40.128596 (+ 2645us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:40.128607 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:40.128647 (+ 40us) server_negotiation.cc:407] Connection header received
0811 19:02:40.129878 (+ 1231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:40.129901 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:40.130233 (+ 332us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:40.131463 (+ 1230us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:02:40.131810 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:40.135542 (+ 3732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:40.138914 (+ 3372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:40.140658 (+ 1744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:40.140957 (+ 299us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:40.141489 (+ 532us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:40.141846 (+ 357us) server_negotiation.cc:299] Negotiation successful
0811 19:02:40.142043 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2461,"thread_start_us":2234,"threads_started":1}
I20260811 19:02:40.142562 28216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:40.125776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.131:39003 (local address 127.0.0.1:59890)
0811 19:02:40.127532 (+ 1756us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:40.127586 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 19:02:40.127809 (+ 223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:02:40.130582 (+ 2773us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:02:40.130593 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:02:40.130638 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:02:40.131182 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:40.131199 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:40.135713 (+ 4514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:40.135719 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:40.138521 (+ 2802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:40.138560 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:40.140922 (+ 2362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:40.140931 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:40.141503 (+ 572us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:40.141583 (+ 80us) client_negotiation.cc:769] Sending connection context
0811 19:02:40.141766 (+ 183us) client_negotiation.cc:240] Negotiation successful
0811 19:02:40.142073 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1323,"mutex_wait_us":73,"thread_start_us":2822,"threads_started":1}
I20260811 19:02:40.147491 28218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:40.129051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.130:40083 (local address 127.0.0.1:60268)
0811 19:02:40.136847 (+ 7796us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:40.136882 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0811 19:02:40.137059 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:02:40.138429 (+ 1370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:02:40.138440 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:02:40.138479 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:02:40.139070 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:40.139083 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:40.141303 (+ 2220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:40.141313 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:40.144073 (+ 2760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:40.144094 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:40.146413 (+ 2319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:40.146420 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:40.146774 (+ 354us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:40.146841 (+ 67us) client_negotiation.cc:769] Sending connection context
0811 19:02:40.146930 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 19:02:40.147086 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7594,"mutex_wait_us":5,"thread_start_us":7340,"threads_started":1}
I20260811 19:02:40.146929 28060 tablet_service.cc:1467] Processing CreateTablet for tablet 91c9b22239f64ddcaab1da34b00cf205 (DEFAULT_TABLE table=test-table [id=ff4af00a658e4366bc73e6d7d14c17e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:02:40.147995 28219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:40.129331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60268 (local address 127.29.112.130:40083)
0811 19:02:40.135981 (+ 6650us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:40.135990 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:40.137149 (+ 1159us) server_negotiation.cc:407] Connection header received
0811 19:02:40.137409 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:40.137418 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:40.137705 (+ 287us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:40.137965 (+ 260us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:02:40.139491 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:40.141130 (+ 1639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:40.144451 (+ 3321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:40.146177 (+ 1726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:40.146375 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:40.146629 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:40.147037 (+ 408us) server_negotiation.cc:299] Negotiation successful
0811 19:02:40.147429 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6474,"thread_start_us":6318,"threads_started":1}
I20260811 19:02:40.148093 28213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:40.126353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:41329 (local address 127.0.0.1:37166)
0811 19:02:40.128869 (+ 2516us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:40.128899 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0811 19:02:40.135216 (+ 6317us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:02:40.136773 (+ 1557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:02:40.136778 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:02:40.136811 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:02:40.138071 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:40.138095 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:40.140970 (+ 2875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:40.140976 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:40.144688 (+ 3712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:40.144704 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:40.146782 (+ 2078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:40.146790 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:40.147025 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:40.147060 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 19:02:40.147149 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 19:02:40.147511 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2394,"mutex_wait_us":159,"thread_start_us":1026,"threads_started":1}
I20260811 19:02:40.149024 28060 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91c9b22239f64ddcaab1da34b00cf205. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:02:40.150192 28217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:40.127120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37166 (local address 127.29.112.129:41329)
0811 19:02:40.135624 (+ 8504us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:40.135639 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:40.135695 (+ 56us) server_negotiation.cc:407] Connection header received
0811 19:02:40.135848 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:40.135858 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:40.136442 (+ 584us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:40.136810 (+ 368us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:02:40.138467 (+ 1657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:40.140786 (+ 2319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:40.144979 (+ 4193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:40.146415 (+ 1436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:40.146674 (+ 259us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:40.146903 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:40.148621 (+ 1718us) server_negotiation.cc:299] Negotiation successful
0811 19:02:40.148894 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8234,"thread_start_us":7863,"threads_started":1}
I20260811 19:02:40.152313 27842 tablet_service.cc:1467] Processing CreateTablet for tablet 91c9b22239f64ddcaab1da34b00cf205 (DEFAULT_TABLE table=test-table [id=ff4af00a658e4366bc73e6d7d14c17e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:02:40.153476 27842 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91c9b22239f64ddcaab1da34b00cf205. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:02:40.153615 27600 tablet_service.cc:1467] Processing CreateTablet for tablet 91c9b22239f64ddcaab1da34b00cf205 (DEFAULT_TABLE table=test-table [id=ff4af00a658e4366bc73e6d7d14c17e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:02:40.154954 27600 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91c9b22239f64ddcaab1da34b00cf205. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:02:40.165048 28222 tablet_bootstrap.cc:492] T 91c9b22239f64ddcaab1da34b00cf205 P eaecfc7d38694ae294c422ef6ca7ca3d: Bootstrap starting.
I20260811 19:02:40.166993 28223 tablet_bootstrap.cc:492] T 91c9b22239f64ddcaab1da34b00cf205 P bf87d2e4a0704ac9925ff8bd593fbfb4: Bootstrap starting.
I20260811 19:02:40.169857 28224 tablet_bootstrap.cc:492] T 91c9b22239f64ddcaab1da34b00cf205 P 9b7d6cb727c94e7cb9512e344001b0eb: Bootstrap starting.
I20260811 19:02:40.170624 28223 tablet_bootstrap.cc:654] T 91c9b22239f64ddcaab1da34b00cf205 P bf87d2e4a0704ac9925ff8bd593fbfb4: Neither blocks nor log segments found. Creating new log.
I20260811 19:02:40.170814 28222 tablet_bootstrap.cc:654] T 91c9b22239f64ddcaab1da34b00cf205 P eaecfc7d38694ae294c422ef6ca7ca3d: Neither blocks nor log segments found. Creating new log.
I20260811 19:02:40.171710 28223 log.cc:824] T 91c9b22239f64ddcaab1da34b00cf205 P bf87d2e4a0704ac9925ff8bd593fbfb4: Log is configured to *not* fsync() on all Append() calls
I20260811 19:02:40.172080 28222 log.cc:824] T 91c9b22239f64ddcaab1da34b00cf205 P eaecfc7d38694ae294c422ef6ca7ca3d: Log is configured to *not* fsync() on all Append() calls
I20260811 19:02:40.173573 28224 tablet_bootstrap.cc:654] T 91c9b22239f64ddcaab1da34b00cf205 P 9b7d6cb727c94e7cb9512e344001b0eb: Neither blocks nor log segments found. Creating new log.
I20260811 19:02:40.174521 28224 log.cc:824] T 91c9b22239f64ddcaab1da34b00cf205 P 9b7d6cb727c94e7cb9512e344001b0eb: Log is configured to *not* fsync() on all Append() calls
I20260811 19:02:40.176465 28223 tablet_bootstrap.cc:492] T 91c9b22239f64ddcaab1da34b00cf205 P bf87d2e4a0704ac9925ff8bd593fbfb4: No bootstrap required, opened a new log
I20260811 19:02:40.176787 28223 ts_tablet_manager.cc:1397] T 91c9b22239f64ddcaab1da34b00cf205 P bf87d2e4a0704ac9925ff8bd593fbfb4: Time spent bootstrapping tablet: real 0.010s user 0.009s sys 0.000s
I20260811 19:02:40.176879 28222 tablet_bootstrap.cc:492] T 91c9b22239f64ddcaab1da34b00cf205 P eaecfc7d38694ae294c422ef6ca7ca3d: No bootstrap required, opened a new log
I20260811 19:02:40.177132 28222 ts_tablet_manager.cc:1397] T 91c9b22239f64ddcaab1da34b00cf205 P eaecfc7d38694ae294c422ef6ca7ca3d: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.000s
I20260811 19:02:40.177212 28224 tablet_bootstrap.cc:492] T 91c9b22239f64ddcaab1da34b00cf205 P 9b7d6cb727c94e7cb9512e344001b0eb: No bootstrap required, opened a new log
I20260811 19:02:40.177455 28224 ts_tablet_manager.cc:1397] T 91c9b22239f64ddcaab1da34b00cf205 P 9b7d6cb727c94e7cb9512e344001b0eb: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260811 19:02:40.184943 28224 raft_consensus.cc:348] T 91c9b22239f64ddcaab1da34b00cf205 P 9b7d6cb727c94e7cb9512e344001b0eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eaecfc7d38694ae294c422ef6ca7ca3d" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 39003 } } peers { permanent_uuid: "bf87d2e4a0704ac9925ff8bd593fbfb4" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 40083 } } peers { permanent_uuid: "9b7d6cb727c94e7cb9512e344001b0eb" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 41329 } }
I20260811 19:02:40.185345 28224 raft_consensus.cc:374] T 91c9b22239f64ddcaab1da34b00cf205 P 9b7d6cb727c94e7cb9512e344001b0eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:02:40.185456 28224 raft_consensus.cc:729] T 91c9b22239f64ddcaab1da34b00cf205 P 9b7d6cb727c94e7cb9512e344001b0eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b7d6cb727c94e7cb9512e344001b0eb, State: Initialized, Role: FOLLOWER
I20260811 19:02:40.186133 28224 consensus_queue.cc:260] T 91c9b22239f64ddcaab1da34b00cf205 P 9b7d6cb727c94e7cb9512e344001b0eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eaecfc7d38694ae294c422ef6ca7ca3d" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 39003 } } peers { permanent_uuid: "bf87d2e4a0704ac9925ff8bd593fbfb4" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 40083 } } peers { permanent_uuid: "9b7d6cb727c94e7cb9512e344001b0eb" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 41329 } }
I20260811 19:02:40.187969 28224 ts_tablet_manager.cc:1428] T 91c9b22239f64ddcaab1da34b00cf205 P 9b7d6cb727c94e7cb9512e344001b0eb: Time spent starting tablet: real 0.010s user 0.004s sys 0.007s
I20260811 19:02:40.187752 28222 raft_consensus.cc:348] T 91c9b22239f64ddcaab1da34b00cf205 P eaecfc7d38694ae294c422ef6ca7ca3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eaecfc7d38694ae294c422ef6ca7ca3d" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 39003 } } peers { permanent_uuid: "bf87d2e4a0704ac9925ff8bd593fbfb4" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 40083 } } peers { permanent_uuid: "9b7d6cb727c94e7cb9512e344001b0eb" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 41329 } }
I20260811 19:02:40.187767 28223 raft_consensus.cc:348] T 91c9b22239f64ddcaab1da34b00cf205 P bf87d2e4a0704ac9925ff8bd593fbfb4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eaecfc7d38694ae294c422ef6ca7ca3d" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 39003 } } peers { permanent_uuid: "bf87d2e4a0704ac9925ff8bd593fbfb4" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 40083 } } peers { permanent_uuid: "9b7d6cb727c94e7cb9512e344001b0eb" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 41329 } }
I20260811 19:02:40.188202 28222 raft_consensus.cc:374] T 91c9b22239f64ddcaab1da34b00cf205 P eaecfc7d38694ae294c422ef6ca7ca3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:02:40.188202 28223 raft_consensus.cc:374] T 91c9b22239f64ddcaab1da34b00cf205 P bf87d2e4a0704ac9925ff8bd593fbfb4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:02:40.188305 28222 raft_consensus.cc:729] T 91c9b22239f64ddcaab1da34b00cf205 P eaecfc7d38694ae294c422ef6ca7ca3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eaecfc7d38694ae294c422ef6ca7ca3d, State: Initialized, Role: FOLLOWER
I20260811 19:02:40.188298 28223 raft_consensus.cc:729] T 91c9b22239f64ddcaab1da34b00cf205 P bf87d2e4a0704ac9925ff8bd593fbfb4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf87d2e4a0704ac9925ff8bd593fbfb4, State: Initialized, Role: FOLLOWER
I20260811 19:02:40.188874 28223 consensus_queue.cc:260] T 91c9b22239f64ddcaab1da34b00cf205 P bf87d2e4a0704ac9925ff8bd593fbfb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eaecfc7d38694ae294c422ef6ca7ca3d" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 39003 } } peers { permanent_uuid: "bf87d2e4a0704ac9925ff8bd593fbfb4" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 40083 } } peers { permanent_uuid: "9b7d6cb727c94e7cb9512e344001b0eb" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 41329 } }
I20260811 19:02:40.188882 28222 consensus_queue.cc:260] T 91c9b22239f64ddcaab1da34b00cf205 P eaecfc7d38694ae294c422ef6ca7ca3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eaecfc7d38694ae294c422ef6ca7ca3d" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 39003 } } peers { permanent_uuid: "bf87d2e4a0704ac9925ff8bd593fbfb4" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 40083 } } peers { permanent_uuid: "9b7d6cb727c94e7cb9512e344001b0eb" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 41329 } }
I20260811 19:02:40.190832 27952 heartbeater.cc:503] Master 127.29.112.190:42977 was elected leader, sending a full tablet report...
I20260811 19:02:40.191819 28222 ts_tablet_manager.cc:1428] T 91c9b22239f64ddcaab1da34b00cf205 P eaecfc7d38694ae294c422ef6ca7ca3d: Time spent starting tablet: real 0.015s user 0.014s sys 0.002s
I20260811 19:02:40.192078 28186 heartbeater.cc:503] Master 127.29.112.190:42977 was elected leader, sending a full tablet report...
I20260811 19:02:40.192292 28229 raft_consensus.cc:482] T 91c9b22239f64ddcaab1da34b00cf205 P 9b7d6cb727c94e7cb9512e344001b0eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:02:40.192593 28229 raft_consensus.cc:504] T 91c9b22239f64ddcaab1da34b00cf205 P 9b7d6cb727c94e7cb9512e344001b0eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eaecfc7d38694ae294c422ef6ca7ca3d" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 39003 } } peers { permanent_uuid: "bf87d2e4a0704ac9925ff8bd593fbfb4" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 40083 } } peers { permanent_uuid: "9b7d6cb727c94e7cb9512e344001b0eb" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 41329 } }
I20260811 19:02:40.193773 28223 ts_tablet_manager.cc:1428] T 91c9b22239f64ddcaab1da34b00cf205 P bf87d2e4a0704ac9925ff8bd593fbfb4: Time spent starting tablet: real 0.017s user 0.010s sys 0.004s
I20260811 19:02:40.194621 28229 leader_election.cc:288] T 91c9b22239f64ddcaab1da34b00cf205 P 9b7d6cb727c94e7cb9512e344001b0eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eaecfc7d38694ae294c422ef6ca7ca3d (127.29.112.131:39003), bf87d2e4a0704ac9925ff8bd593fbfb4 (127.29.112.130:40083)
I20260811 19:02:40.203446 28214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:40.196514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.129:43889 (local address 127.29.112.131:39003)
0811 19:02:40.196700 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:40.196705 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:40.196722 (+ 17us) server_negotiation.cc:407] Connection header received
0811 19:02:40.196803 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:40.196809 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:40.196890 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:40.196990 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:02:40.197922 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:40.199391 (+ 1469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:40.201549 (+ 2158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:40.202442 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:40.202717 (+ 275us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:40.202803 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:40.203057 (+ 254us) server_negotiation.cc:299] Negotiation successful
0811 19:02:40.203196 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 19:02:40.203534 28235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:40.195267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.131:39003 (local address 127.29.112.129:43889)
0811 19:02:40.196193 (+ 926us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:40.196216 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 19:02:40.196346 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:02:40.197233 (+ 887us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:02:40.197237 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:02:40.197264 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:02:40.197759 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:40.197774 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:40.199600 (+ 1826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:40.199605 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:40.201338 (+ 1733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:40.201354 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:40.202746 (+ 1392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:40.202751 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:40.202895 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:40.202920 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 19:02:40.203000 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 19:02:40.203210 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":734,"mutex_wait_us":93,"spinlock_wait_cycles":7424,"thread_start_us":686,"threads_started":1}
I20260811 19:02:40.204680 28100 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91c9b22239f64ddcaab1da34b00cf205" candidate_uuid: "9b7d6cb727c94e7cb9512e344001b0eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eaecfc7d38694ae294c422ef6ca7ca3d" is_pre_election: true
I20260811 19:02:40.205175 28100 raft_consensus.cc:2454] T 91c9b22239f64ddcaab1da34b00cf205 P eaecfc7d38694ae294c422ef6ca7ca3d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b7d6cb727c94e7cb9512e344001b0eb in term 0.
I20260811 19:02:40.205858 27516 leader_election.cc:302] T 91c9b22239f64ddcaab1da34b00cf205 P 9b7d6cb727c94e7cb9512e344001b0eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9b7d6cb727c94e7cb9512e344001b0eb, eaecfc7d38694ae294c422ef6ca7ca3d; no voters:
I20260811 19:02:40.206283 28229 raft_consensus.cc:2781] T 91c9b22239f64ddcaab1da34b00cf205 P 9b7d6cb727c94e7cb9512e344001b0eb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 19:02:40.206442 28219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:40.197030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.129:34763 (local address 127.29.112.130:40083)
0811 19:02:40.199156 (+ 2126us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:40.199166 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:40.200428 (+ 1262us) server_negotiation.cc:407] Connection header received
0811 19:02:40.200790 (+ 362us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:40.200796 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:40.200981 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:40.201122 (+ 141us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:02:40.201789 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:40.202457 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:40.204670 (+ 2213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:40.205549 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:40.205706 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:40.205773 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:40.206071 (+ 298us) server_negotiation.cc:299] Negotiation successful
0811 19:02:40.206190 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114}
I20260811 19:02:40.206462 28229 raft_consensus.cc:482] T 91c9b22239f64ddcaab1da34b00cf205 P 9b7d6cb727c94e7cb9512e344001b0eb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 19:02:40.206594 28229 raft_consensus.cc:3037] T 91c9b22239f64ddcaab1da34b00cf205 P 9b7d6cb727c94e7cb9512e344001b0eb [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:02:40.207109 28236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:40.196677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.130:40083 (local address 127.29.112.129:34763)
0811 19:02:40.200255 (+ 3578us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:40.200278 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 19:02:40.200581 (+ 303us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:02:40.201132 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:02:40.201137 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:02:40.201165 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:02:40.201609 (+ 444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:40.201624 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:40.202669 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:40.202674 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:40.204490 (+ 1816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:40.204504 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:40.205743 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:40.205749 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:40.205898 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:40.205920 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 19:02:40.206042 (+ 122us) client_negotiation.cc:240] Negotiation successful
0811 19:02:40.206166 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3412,"thread_start_us":3044,"threads_started":1}
I20260811 19:02:40.209002 28229 raft_consensus.cc:504] T 91c9b22239f64ddcaab1da34b00cf205 P 9b7d6cb727c94e7cb9512e344001b0eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eaecfc7d38694ae294c422ef6ca7ca3d" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 39003 } } peers { permanent_uuid: "bf87d2e4a0704ac9925ff8bd593fbfb4" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 40083 } } peers { permanent_uuid: "9b7d6cb727c94e7cb9512e344001b0eb" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 41329 } }
I20260811 19:02:40.209600 28229 leader_election.cc:288] T 91c9b22239f64ddcaab1da34b00cf205 P 9b7d6cb727c94e7cb9512e344001b0eb [CANDIDATE]: Term 1 election: Requested vote from peers eaecfc7d38694ae294c422ef6ca7ca3d (127.29.112.131:39003), bf87d2e4a0704ac9925ff8bd593fbfb4 (127.29.112.130:40083)
I20260811 19:02:40.210237 28100 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91c9b22239f64ddcaab1da34b00cf205" candidate_uuid: "9b7d6cb727c94e7cb9512e344001b0eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eaecfc7d38694ae294c422ef6ca7ca3d"
I20260811 19:02:40.210439 28100 raft_consensus.cc:3037] T 91c9b22239f64ddcaab1da34b00cf205 P eaecfc7d38694ae294c422ef6ca7ca3d [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:02:40.210860 27873 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91c9b22239f64ddcaab1da34b00cf205" candidate_uuid: "9b7d6cb727c94e7cb9512e344001b0eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf87d2e4a0704ac9925ff8bd593fbfb4" is_pre_election: true
I20260811 19:02:40.210942 27872 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91c9b22239f64ddcaab1da34b00cf205" candidate_uuid: "9b7d6cb727c94e7cb9512e344001b0eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf87d2e4a0704ac9925ff8bd593fbfb4"
I20260811 19:02:40.211338 27873 raft_consensus.cc:2454] T 91c9b22239f64ddcaab1da34b00cf205 P bf87d2e4a0704ac9925ff8bd593fbfb4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b7d6cb727c94e7cb9512e344001b0eb in term 0.
I20260811 19:02:40.213299 28100 raft_consensus.cc:2454] T 91c9b22239f64ddcaab1da34b00cf205 P eaecfc7d38694ae294c422ef6ca7ca3d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b7d6cb727c94e7cb9512e344001b0eb in term 1.
I20260811 19:02:40.213723 27516 leader_election.cc:302] T 91c9b22239f64ddcaab1da34b00cf205 P 9b7d6cb727c94e7cb9512e344001b0eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9b7d6cb727c94e7cb9512e344001b0eb, eaecfc7d38694ae294c422ef6ca7ca3d; no voters: bf87d2e4a0704ac9925ff8bd593fbfb4
I20260811 19:02:40.214042 28229 raft_consensus.cc:2781] T 91c9b22239f64ddcaab1da34b00cf205 P 9b7d6cb727c94e7cb9512e344001b0eb [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:02:40.214591 28229 raft_consensus.cc:686] T 91c9b22239f64ddcaab1da34b00cf205 P 9b7d6cb727c94e7cb9512e344001b0eb [term 1 LEADER]: Becoming Leader. State: Replica: 9b7d6cb727c94e7cb9512e344001b0eb, State: Running, Role: LEADER
I20260811 19:02:40.215119 28229 consensus_queue.cc:237] T 91c9b22239f64ddcaab1da34b00cf205 P 9b7d6cb727c94e7cb9512e344001b0eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eaecfc7d38694ae294c422ef6ca7ca3d" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 39003 } } peers { permanent_uuid: "bf87d2e4a0704ac9925ff8bd593fbfb4" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 40083 } } peers { permanent_uuid: "9b7d6cb727c94e7cb9512e344001b0eb" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 41329 } }
I20260811 19:02:40.221422 27368 catalog_manager.cc:5495] T 91c9b22239f64ddcaab1da34b00cf205 P 9b7d6cb727c94e7cb9512e344001b0eb reported cstate change: term changed from 0 to 1, leader changed from <none> to 9b7d6cb727c94e7cb9512e344001b0eb (127.29.112.129). New cstate: current_term: 1 leader_uuid: "9b7d6cb727c94e7cb9512e344001b0eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eaecfc7d38694ae294c422ef6ca7ca3d" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf87d2e4a0704ac9925ff8bd593fbfb4" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 40083 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9b7d6cb727c94e7cb9512e344001b0eb" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 41329 } health_report { overall_health: HEALTHY } } }
I20260811 19:02:40.244170 28217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:40.237119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37182 (local address 127.29.112.129:41329)
0811 19:02:40.237388 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:40.237397 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:40.237674 (+ 277us) server_negotiation.cc:407] Connection header received
0811 19:02:40.237832 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:40.237837 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:40.237956 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:40.238112 (+ 156us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:02:40.238961 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:40.239687 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:40.241549 (+ 1862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:40.242086 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:40.242203 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:40.242835 (+ 632us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:02:40.243384 (+ 549us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:02:40.243512 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:40.243819 (+ 307us) server_negotiation.cc:299] Negotiation successful
0811 19:02:40.243907 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260811 19:02:40.253194 27872 raft_consensus.cc:3037] T 91c9b22239f64ddcaab1da34b00cf205 P bf87d2e4a0704ac9925ff8bd593fbfb4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:02:40.253527 28100 raft_consensus.cc:1261] T 91c9b22239f64ddcaab1da34b00cf205 P eaecfc7d38694ae294c422ef6ca7ca3d [term 1 FOLLOWER]: Refusing update from remote peer 9b7d6cb727c94e7cb9512e344001b0eb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:02:40.254794 28241 consensus_queue.cc:1035] T 91c9b22239f64ddcaab1da34b00cf205 P 9b7d6cb727c94e7cb9512e344001b0eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "eaecfc7d38694ae294c422ef6ca7ca3d" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:02:40.256659 27872 raft_consensus.cc:1261] T 91c9b22239f64ddcaab1da34b00cf205 P bf87d2e4a0704ac9925ff8bd593fbfb4 [term 1 FOLLOWER]: Refusing update from remote peer 9b7d6cb727c94e7cb9512e344001b0eb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:02:40.257903 28241 consensus_queue.cc:1035] T 91c9b22239f64ddcaab1da34b00cf205 P 9b7d6cb727c94e7cb9512e344001b0eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf87d2e4a0704ac9925ff8bd593fbfb4" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 40083 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 19:02:40.263803 27726 tablet.cc:2369] T 91c9b22239f64ddcaab1da34b00cf205 P 9b7d6cb727c94e7cb9512e344001b0eb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 19:02:40.270624 28244 mvcc.cc:205] Tried to move back new op lower bound from 7317401437187989504 to 7317401437045039104. Current Snapshot: MvccSnapshot[applied={T|T < 7317401437187989504}]
I20260811 19:02:40.278455 28253 mvcc.cc:205] Tried to move back new op lower bound from 7317401437187989504 to 7317401437045039104. Current Snapshot: MvccSnapshot[applied={T|T < 7317401437187989504}]
I20260811 19:02:40.294808 27368 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39460:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:02:40.295295 27368 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39460:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:02:40.299906 27368 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3b6045caba42449daf034232f450bbe6: Sending DeleteTablet for 3 replicas of tablet 91c9b22239f64ddcaab1da34b00cf205
I20260811 19:02:40.301174 28060 tablet_service.cc:1514] Processing DeleteTablet for tablet 91c9b22239f64ddcaab1da34b00cf205 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:02:40 UTC) from {username='slave'} at 127.0.0.1:59890
I20260811 19:02:40.301226 27842 tablet_service.cc:1514] Processing DeleteTablet for tablet 91c9b22239f64ddcaab1da34b00cf205 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:02:40 UTC) from {username='slave'} at 127.0.0.1:60268
I20260811 19:02:40.303194 27600 tablet_service.cc:1514] Processing DeleteTablet for tablet 91c9b22239f64ddcaab1da34b00cf205 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:02:40 UTC) from {username='slave'} at 127.0.0.1:37166
I20260811 19:02:40.303522 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 27490
I20260811 19:02:40.303530 28267 tablet_replica.cc:330] T 91c9b22239f64ddcaab1da34b00cf205 P eaecfc7d38694ae294c422ef6ca7ca3d: stopping tablet replica
I20260811 19:02:40.304061 28267 raft_consensus.cc:2229] T 91c9b22239f64ddcaab1da34b00cf205 P eaecfc7d38694ae294c422ef6ca7ca3d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:02:40.304646 28267 raft_consensus.cc:2258] T 91c9b22239f64ddcaab1da34b00cf205 P eaecfc7d38694ae294c422ef6ca7ca3d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:02:40.305612 28266 tablet_replica.cc:330] T 91c9b22239f64ddcaab1da34b00cf205 P bf87d2e4a0704ac9925ff8bd593fbfb4: stopping tablet replica
I20260811 19:02:40.306075 28266 raft_consensus.cc:2229] T 91c9b22239f64ddcaab1da34b00cf205 P bf87d2e4a0704ac9925ff8bd593fbfb4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:02:40.306603 28266 raft_consensus.cc:2258] T 91c9b22239f64ddcaab1da34b00cf205 P bf87d2e4a0704ac9925ff8bd593fbfb4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:02:40.309208 28267 ts_tablet_manager.cc:1905] T 91c9b22239f64ddcaab1da34b00cf205 P eaecfc7d38694ae294c422ef6ca7ca3d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:02:40.309885 28268 tablet_replica.cc:330] T 91c9b22239f64ddcaab1da34b00cf205 P 9b7d6cb727c94e7cb9512e344001b0eb: stopping tablet replica
I20260811 19:02:40.310338 28268 raft_consensus.cc:2229] T 91c9b22239f64ddcaab1da34b00cf205 P 9b7d6cb727c94e7cb9512e344001b0eb [term 1 LEADER]: Raft consensus shutting down.
I20260811 19:02:40.311147 28268 raft_consensus.cc:2258] T 91c9b22239f64ddcaab1da34b00cf205 P 9b7d6cb727c94e7cb9512e344001b0eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:02:40.312573 28266 ts_tablet_manager.cc:1905] T 91c9b22239f64ddcaab1da34b00cf205 P bf87d2e4a0704ac9925ff8bd593fbfb4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:02:40.318055 28267 ts_tablet_manager.cc:1918] T 91c9b22239f64ddcaab1da34b00cf205 P eaecfc7d38694ae294c422ef6ca7ca3d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:02:40.318295 28267 log.cc:1196] T 91c9b22239f64ddcaab1da34b00cf205 P eaecfc7d38694ae294c422ef6ca7ca3d: Deleting WAL directory at /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474615205622-30146-0/minicluster-data/ts-2/wal/wals/91c9b22239f64ddcaab1da34b00cf205
I20260811 19:02:40.318982 28267 ts_tablet_manager.cc:1939] T 91c9b22239f64ddcaab1da34b00cf205 P eaecfc7d38694ae294c422ef6ca7ca3d: Deleting consensus metadata
I20260811 19:02:40.320655 28268 ts_tablet_manager.cc:1905] T 91c9b22239f64ddcaab1da34b00cf205 P 9b7d6cb727c94e7cb9512e344001b0eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:02:40.320861 27347 catalog_manager.cc:4841] TS eaecfc7d38694ae294c422ef6ca7ca3d (127.29.112.131:39003): tablet 91c9b22239f64ddcaab1da34b00cf205 (table test-table [id=ff4af00a658e4366bc73e6d7d14c17e7]) successfully deleted
I20260811 19:02:40.323094 28266 ts_tablet_manager.cc:1918] T 91c9b22239f64ddcaab1da34b00cf205 P bf87d2e4a0704ac9925ff8bd593fbfb4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:02:40.323280 28266 log.cc:1196] T 91c9b22239f64ddcaab1da34b00cf205 P bf87d2e4a0704ac9925ff8bd593fbfb4: Deleting WAL directory at /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474615205622-30146-0/minicluster-data/ts-1/wal/wals/91c9b22239f64ddcaab1da34b00cf205
I20260811 19:02:40.323863 28266 ts_tablet_manager.cc:1939] T 91c9b22239f64ddcaab1da34b00cf205 P bf87d2e4a0704ac9925ff8bd593fbfb4: Deleting consensus metadata
I20260811 19:02:40.324680 28217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:40.305018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37188 (local address 127.29.112.129:41329)
0811 19:02:40.305238 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:40.305245 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:40.315659 (+ 10414us) server_negotiation.cc:407] Connection header received
0811 19:02:40.316138 (+ 479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:40.316144 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:40.316243 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:40.316319 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:02:40.317004 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:40.318338 (+ 1334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:40.320353 (+ 2015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:40.320990 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:40.321096 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:40.322657 (+ 1561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:02:40.322671 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:02:40.322687 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:02:40.323708 (+ 1021us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:02:40.323769 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:02:40.323795 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:02:40.323923 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:40.324202 (+ 279us) server_negotiation.cc:299] Negotiation successful
0811 19:02:40.324306 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 19:02:40.325516 27715 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:02:40.325903 27347 catalog_manager.cc:4841] TS bf87d2e4a0704ac9925ff8bd593fbfb4 (127.29.112.130:40083): tablet 91c9b22239f64ddcaab1da34b00cf205 (table test-table [id=ff4af00a658e4366bc73e6d7d14c17e7]) successfully deleted
I20260811 19:02:40.441695 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 27490
W20260811 19:02:40.480432 27349 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.112.129:41329 (error 108)
I20260811 19:02:40.481451 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 27730
I20260811 19:02:40.482985 28213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:40.482376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:41329 (local address 127.0.0.1:37200)
0811 19:02:40.482656 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:40.482786 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.129:41329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137}
W20260811 19:02:40.483213 27349 catalog_manager.cc:4584] TS 9b7d6cb727c94e7cb9512e344001b0eb (127.29.112.129:41329): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 91c9b22239f64ddcaab1da34b00cf205: Network error: Client connection negotiation failed: client connection to 127.29.112.129:41329: connect: Connection refused (error 111)
I20260811 19:02:40.489529 28219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:40.482427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60282 (local address 127.29.112.130:40083)
0811 19:02:40.482690 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:40.482695 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:40.482716 (+ 21us) server_negotiation.cc:407] Connection header received
0811 19:02:40.482934 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:40.482938 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:40.483031 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:40.483159 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:02:40.483804 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:40.484699 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:40.486125 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:40.486669 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:40.486830 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:40.488045 (+ 1215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:02:40.488070 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:02:40.488084 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:02:40.488672 (+ 588us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:02:40.488719 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:02:40.488740 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:02:40.488884 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:40.489034 (+ 150us) server_negotiation.cc:299] Negotiation successful
0811 19:02:40.489173 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136}
I20260811 19:02:40.490302 27944 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:02:40.537127 28213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:40.536601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:41329 (local address 127.0.0.1:37204)
0811 19:02:40.536855 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:40.536961 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.129:41329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
I20260811 19:02:40.538758 28213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:40.538121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:41329 (local address 127.0.0.1:37210)
0811 19:02:40.538501 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:40.538632 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.129:41329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":258}
I20260811 19:02:40.589013 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 27730
I20260811 19:02:40.599611 28213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:40.599108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:41329 (local address 127.0.0.1:37214)
0811 19:02:40.599320 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:40.599420 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.129:41329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260811 19:02:40.600677 28213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:40.600355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:41329 (local address 127.0.0.1:37220)
0811 19:02:40.600503 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:40.600569 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.129:41329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260811 19:02:40.615485 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 27956
I20260811 19:02:40.624603 28214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:40.616677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59892 (local address 127.29.112.131:39003)
0811 19:02:40.616874 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:40.616879 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:40.616901 (+ 22us) server_negotiation.cc:407] Connection header received
0811 19:02:40.616973 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:40.616977 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:40.617086 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:40.617192 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:02:40.617722 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:40.618950 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:40.620722 (+ 1772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:40.621145 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:40.621256 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:40.622831 (+ 1575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:02:40.622845 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:02:40.622866 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:02:40.623806 (+ 940us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:02:40.623867 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:02:40.623890 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:02:40.623996 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:40.624196 (+ 200us) server_negotiation.cc:299] Negotiation successful
0811 19:02:40.624295 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 19:02:40.625648 28179 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:02:40.681485 28213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:40.680872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:41329 (local address 127.0.0.1:37228)
0811 19:02:40.681137 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:40.681286 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.129:41329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260811 19:02:40.683271 28213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:40.682613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:41329 (local address 127.0.0.1:37244)
0811 19:02:40.682956 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:40.683074 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.129:41329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133}
I20260811 19:02:40.723428 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 27956
I20260811 19:02:40.754281 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 27318
I20260811 19:02:40.755633 27433 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:02:40.885836 28213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:40.884963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:41329 (local address 127.0.0.1:37250)
0811 19:02:40.885258 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:40.885419 (+ 161us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.129:41329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260811 19:02:40.886611 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 27318
2026-08-11T19:02:40Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (3221 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:02:41 dist-test-slave-h6n0 krb5kdc[28278](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:02:41 dist-test-slave-h6n0 krb5kdc[28278](info): set up 2 sockets
Aug 11 19:02:41 dist-test-slave-h6n0 krb5kdc[28278](info): commencing operation
krb5kdc: starting...
W20260811 19:02:42.504351 30146 mini_kdc.cc:121] Time spent starting KDC: real 1.553s user 0.000s sys 0.056s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/krb5kdc/test-user.keytab.
Aug 11 19:02:42 dist-test-slave-h6n0 krb5kdc[28278](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474962, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:02:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:02:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.112.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab.
Entry for principal kudu/127.29.112.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab.
W20260811 19:02:42.945066 30146 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.29.112.190: real 0.103s user 0.000s sys 0.035s
I20260811 19:02:42.946139 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.190:35795
--webserver_interface=127.29.112.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab
--principal=kudu/127.29.112.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:35575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.190:35795
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:02:43.443451 28298 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:02:43.444451 28298 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:02:43.444906 28298 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:02:43.456038 28298 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:02:43.456193 28298 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:02:43.456302 28298 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:02:43.456416 28298 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:02:43.456463 28298 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:02:43.481094 28298 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:35575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.112.190:35795
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab
--principal=kudu/127.29.112.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.190:35795
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.112.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:02:43.483784 28298 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:02:43.486192 28298 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:02:43.511135 28306 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:02:43.512849 28314 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:02:43.513469 28307 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:02:43.514230 28298 server_base.cc:1034] running on GCE node
I20260811 19:02:43.516077 28298 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:02:43.518729 28298 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:02:43.520056 28298 hybrid_clock.cc:648] HybridClock initialized: now 1786474963519970 us; error 61 us; skew 500 ppm
I20260811 19:02:43.520640 28298 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:02:43 dist-test-slave-h6n0 krb5kdc[28278](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474963, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:02:43.526018 28298 init.cc:377] Logged in from keytab as kudu/127.29.112.190@KRBTEST.COM (short username kudu)
I20260811 19:02:43.528517 28298 webserver.cc:460] Webserver started at http://127.29.112.190:41837/ using document root <none> and password file <none>
I20260811 19:02:43.529615 28298 fs_manager.cc:359] Metadata directory not provided
I20260811 19:02:43.529798 28298 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:02:43.530218 28298 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:02:43.535930 28298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/minicluster-data/master-0/data/instance:
uuid: "fb4a48b2a3d148129d73bf9f06451002"
format_stamp: "Formatted at 2026-08-11 19:02:43 on dist-test-slave-h6n0"
server_key: "d5c18ace4d0f4964275d960b6e065107"
server_key_iv: "ca304224646f132ab7f99d9e33b1b6a4"
server_key_version: "encryptionkey@0"
I20260811 19:02:43.537132 28298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/minicluster-data/master-0/wal/instance:
uuid: "fb4a48b2a3d148129d73bf9f06451002"
format_stamp: "Formatted at 2026-08-11 19:02:43 on dist-test-slave-h6n0"
server_key: "d5c18ace4d0f4964275d960b6e065107"
server_key_iv: "ca304224646f132ab7f99d9e33b1b6a4"
server_key_version: "encryptionkey@0"
I20260811 19:02:43.547261 28298 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.011s sys 0.000s
I20260811 19:02:43.552642 28329 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:43.554909 28298 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.007s sys 0.000s
I20260811 19:02:43.555212 28298 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/minicluster-data/master-0/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/minicluster-data/master-0/wal
uuid: "fb4a48b2a3d148129d73bf9f06451002"
format_stamp: "Formatted at 2026-08-11 19:02:43 on dist-test-slave-h6n0"
server_key: "d5c18ace4d0f4964275d960b6e065107"
server_key_iv: "ca304224646f132ab7f99d9e33b1b6a4"
server_key_version: "encryptionkey@0"
I20260811 19:02:43.555500 28298 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:02:43.588524 28298 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:02:43.594480 28298 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:02:43.594993 28298 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:02:43.622128 28298 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.190:35795
I20260811 19:02:43.622131 28427 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.190:35795 every 8 connection(s)
I20260811 19:02:43.624092 28298 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
I20260811 19:02:43.626369 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 28298
I20260811 19:02:43.626688 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/minicluster-data/master-0/wal/instance
I20260811 19:02:43.627228 30146 external_mini_cluster.cc:1428] Setting key ffeba0e46725634e0d77bc21442c7b2d
I20260811 19:02:43.632609 28429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 19:02:43 dist-test-slave-h6n0 krb5kdc[28278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474962, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 19:02:43.649183 28429 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb4a48b2a3d148129d73bf9f06451002: Bootstrap starting.
I20260811 19:02:43.654359 28429 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fb4a48b2a3d148129d73bf9f06451002: Neither blocks nor log segments found. Creating new log.
I20260811 19:02:43.656148 28429 log.cc:824] T 00000000000000000000000000000000 P fb4a48b2a3d148129d73bf9f06451002: Log is configured to *not* fsync() on all Append() calls
I20260811 19:02:43.660670 28429 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb4a48b2a3d148129d73bf9f06451002: No bootstrap required, opened a new log
I20260811 19:02:43.660714 28433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:43.629673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39050 (local address 127.29.112.190:35795)
0811 19:02:43.630773 (+ 1100us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:43.630789 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:43.630850 (+ 61us) server_negotiation.cc:407] Connection header received
0811 19:02:43.633098 (+ 2248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:43.633140 (+ 42us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:43.633883 (+ 743us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:43.634610 (+ 727us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:02:43.637035 (+ 2425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:43.639036 (+ 2001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:43.640700 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:43.642625 (+ 1925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:43.642864 (+ 239us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:43.649392 (+ 6528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:02:43.649405 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:02:43.649429 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:02:43.649487 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:02:43.654350 (+ 4863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:43.655449 (+ 1099us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:43.655457 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:43.655465 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:43.655706 (+ 241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:43.656195 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:43.656199 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:43.656202 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:43.657029 (+ 827us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:02:43.657625 (+ 596us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:43.658418 (+ 793us) server_negotiation.cc:299] Negotiation successful
0811 19:02:43.659238 (+ 820us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":657,"thread_start_us":409,"threads_started":1}
I20260811 19:02:43.670085 28429 raft_consensus.cc:348] T 00000000000000000000000000000000 P fb4a48b2a3d148129d73bf9f06451002 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb4a48b2a3d148129d73bf9f06451002" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 35795 } }
I20260811 19:02:43.670634 28429 raft_consensus.cc:374] T 00000000000000000000000000000000 P fb4a48b2a3d148129d73bf9f06451002 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:02:43.670789 28429 raft_consensus.cc:729] T 00000000000000000000000000000000 P fb4a48b2a3d148129d73bf9f06451002 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb4a48b2a3d148129d73bf9f06451002, State: Initialized, Role: FOLLOWER
I20260811 19:02:43.671535 28429 consensus_queue.cc:260] T 00000000000000000000000000000000 P fb4a48b2a3d148129d73bf9f06451002 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb4a48b2a3d148129d73bf9f06451002" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 35795 } }
I20260811 19:02:43.671825 28429 raft_consensus.cc:388] T 00000000000000000000000000000000 P fb4a48b2a3d148129d73bf9f06451002 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:02:43.672163 28429 raft_consensus.cc:482] T 00000000000000000000000000000000 P fb4a48b2a3d148129d73bf9f06451002 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:02:43.672622 28429 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fb4a48b2a3d148129d73bf9f06451002 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:02:43.677423 28429 raft_consensus.cc:504] T 00000000000000000000000000000000 P fb4a48b2a3d148129d73bf9f06451002 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb4a48b2a3d148129d73bf9f06451002" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 35795 } }
I20260811 19:02:43.678107 28429 leader_election.cc:302] T 00000000000000000000000000000000 P fb4a48b2a3d148129d73bf9f06451002 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fb4a48b2a3d148129d73bf9f06451002; no voters:
I20260811 19:02:43.679353 28429 leader_election.cc:288] T 00000000000000000000000000000000 P fb4a48b2a3d148129d73bf9f06451002 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:02:43.679454 28436 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fb4a48b2a3d148129d73bf9f06451002 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:02:43.681083 28436 raft_consensus.cc:686] T 00000000000000000000000000000000 P fb4a48b2a3d148129d73bf9f06451002 [term 1 LEADER]: Becoming Leader. State: Replica: fb4a48b2a3d148129d73bf9f06451002, State: Running, Role: LEADER
I20260811 19:02:43.682003 28429 sys_catalog.cc:564] T 00000000000000000000000000000000 P fb4a48b2a3d148129d73bf9f06451002 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:02:43.681893 28436 consensus_queue.cc:237] T 00000000000000000000000000000000 P fb4a48b2a3d148129d73bf9f06451002 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb4a48b2a3d148129d73bf9f06451002" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 35795 } }
I20260811 19:02:43.685946 28438 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb4a48b2a3d148129d73bf9f06451002 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fb4a48b2a3d148129d73bf9f06451002. Latest consensus state: current_term: 1 leader_uuid: "fb4a48b2a3d148129d73bf9f06451002" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb4a48b2a3d148129d73bf9f06451002" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 35795 } } }
I20260811 19:02:43.686585 28438 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb4a48b2a3d148129d73bf9f06451002 [sys.catalog]: This master's current role is: LEADER
I20260811 19:02:43.687230 28437 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb4a48b2a3d148129d73bf9f06451002 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fb4a48b2a3d148129d73bf9f06451002" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb4a48b2a3d148129d73bf9f06451002" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 35795 } } }
I20260811 19:02:43.687816 28437 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb4a48b2a3d148129d73bf9f06451002 [sys.catalog]: This master's current role is: LEADER
I20260811 19:02:43.691341 28446 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:02:43.699734 28446 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:02:43.714926 28446 catalog_manager.cc:1347] Generated new cluster ID: 146456c90bf44fa3b42f3f1437f66341
I20260811 19:02:43.715294 28446 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:02:43.746578 28446 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:02:43.748554 28446 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:02:43.758028 28446 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fb4a48b2a3d148129d73bf9f06451002: Generated new TSK 0
I20260811 19:02:43.759307 28446 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.112.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.129.keytab.
Entry for principal kudu/127.29.112.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.129.keytab.
W20260811 19:02:43.875360 30146 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.29.112.129: real 0.101s user 0.004s sys 0.034s
I20260811 19:02:43.876400 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.129:0
--local_ip_for_outbound_sockets=127.29.112.129
--webserver_interface=127.29.112.129
--webserver_port=0
--tserver_master_addrs=127.29.112.190:35795
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.129.keytab
--principal=kudu/127.29.112.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:35575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:02:44.387425 28470 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:02:44.387773 28470 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:02:44.387884 28470 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:02:44.396974 28470 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:02:44.397133 28470 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:02:44.397327 28470 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.129
I20260811 19:02:44.415632 28470 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:35575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.129.keytab
--principal=kudu/127.29.112.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.112.129
--webserver_port=0
--tserver_master_addrs=127.29.112.190:35795
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.129
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:02:44.417200 28470 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:02:44.418722 28470 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:02:44.440999 28479 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:02:44.441619 28480 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:02:44.441923 28482 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:02:44.443780 28470 server_base.cc:1034] running on GCE node
I20260811 19:02:44.444706 28470 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:02:44.445855 28470 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:02:44.447165 28470 hybrid_clock.cc:648] HybridClock initialized: now 1786474964447108 us; error 39 us; skew 500 ppm
I20260811 19:02:44.447655 28470 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:02:44 dist-test-slave-h6n0 krb5kdc[28278](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474964, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:02:44.454895 28470 init.cc:377] Logged in from keytab as kudu/127.29.112.129@KRBTEST.COM (short username kudu)
I20260811 19:02:44.459755 28470 webserver.cc:460] Webserver started at http://127.29.112.129:40345/ using document root <none> and password file <none>
I20260811 19:02:44.460736 28470 fs_manager.cc:359] Metadata directory not provided
I20260811 19:02:44.460868 28470 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:02:44.461278 28470 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:02:44.465644 28470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/minicluster-data/ts-0/data/instance:
uuid: "6399a55532ef4ee382a1a3111bf92a85"
format_stamp: "Formatted at 2026-08-11 19:02:44 on dist-test-slave-h6n0"
server_key: "0a73e830a2e3464dfa7bd75cc68af2a3"
server_key_iv: "81479542e23d97c871f23b8f50f582d6"
server_key_version: "encryptionkey@0"
I20260811 19:02:44.466499 28470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/minicluster-data/ts-0/wal/instance:
uuid: "6399a55532ef4ee382a1a3111bf92a85"
format_stamp: "Formatted at 2026-08-11 19:02:44 on dist-test-slave-h6n0"
server_key: "0a73e830a2e3464dfa7bd75cc68af2a3"
server_key_iv: "81479542e23d97c871f23b8f50f582d6"
server_key_version: "encryptionkey@0"
I20260811 19:02:44.473994 28470 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.002s sys 0.004s
I20260811 19:02:44.479008 28498 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:44.481952 28470 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.004s sys 0.000s
I20260811 19:02:44.482400 28470 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/minicluster-data/ts-0/wal
uuid: "6399a55532ef4ee382a1a3111bf92a85"
format_stamp: "Formatted at 2026-08-11 19:02:44 on dist-test-slave-h6n0"
server_key: "0a73e830a2e3464dfa7bd75cc68af2a3"
server_key_iv: "81479542e23d97c871f23b8f50f582d6"
server_key_version: "encryptionkey@0"
I20260811 19:02:44.483053 28470 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:02:44.508320 28470 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:02:44.517531 28470 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:02:44.518026 28470 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:02:44.520159 28470 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:02:44.521966 28470 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:02:44.522060 28470 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:44.522682 28470 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:02:44.522789 28470 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:44.574867 28470 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.129:39721
I20260811 19:02:44.574896 28702 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.129:39721 every 8 connection(s)
I20260811 19:02:44.576444 28470 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/minicluster-data/ts-0/data/info.pb
I20260811 19:02:44.581097 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 28470
I20260811 19:02:44.581329 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/minicluster-data/ts-0/wal/instance
I20260811 19:02:44.581764 30146 external_mini_cluster.cc:1428] Setting key 2059c21a88c96c67d051fd76eca0d889
Aug 11 19:02:44 dist-test-slave-h6n0 krb5kdc[28278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474964, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.129@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 19:02:44.605211 28705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:44.579798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.129:49911 (local address 127.29.112.190:35795)
0811 19:02:44.580600 (+ 802us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:44.580605 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:44.581167 (+ 562us) server_negotiation.cc:407] Connection header received
0811 19:02:44.583901 (+ 2734us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:44.583907 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:44.584013 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:44.584149 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:02:44.586018 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:44.586967 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:44.588791 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:44.589324 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:44.589684 (+ 360us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:44.595454 (+ 5770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:02:44.595460 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:02:44.595463 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:02:44.595504 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:02:44.600341 (+ 4837us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:44.601718 (+ 1377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:44.601724 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:44.601726 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:44.601895 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:44.602607 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:44.602615 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:44.602618 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:44.603090 (+ 472us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:02:44.603296 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:44.604544 (+ 1248us) server_negotiation.cc:299] Negotiation successful
0811 19:02:44.604841 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":650,"thread_start_us":501,"threads_started":1}
I20260811 19:02:44.606643 28706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:44.580047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:35795 (local address 127.29.112.129:49911)
0811 19:02:44.580971 (+ 924us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:44.581019 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 19:02:44.583579 (+ 2560us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:02:44.584447 (+ 868us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:02:44.584456 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:02:44.585092 (+ 636us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:02:44.585779 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:44.585800 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:44.587157 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:44.587162 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:44.588610 (+ 1448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:44.588625 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:44.589839 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:44.589848 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:44.590288 (+ 440us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:44.591517 (+ 1229us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:02:44.591558 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:02:44.595137 (+ 3579us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:02:44.600709 (+ 5572us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:02:44.600722 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:02:44.600744 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:02:44.601427 (+ 683us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:02:44.602039 (+ 612us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:02:44.602047 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:02:44.602054 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:02:44.602334 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:02:44.603298 (+ 964us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:02:44.603309 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:02:44.603864 (+ 555us) client_negotiation.cc:769] Sending connection context
0811 19:02:44.604249 (+ 385us) client_negotiation.cc:240] Negotiation successful
0811 19:02:44.604951 (+ 702us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":640,"thread_start_us":383,"threads_started":1}
I20260811 19:02:44.609031 28703 heartbeater.cc:347] Connected to a master server at 127.29.112.190:35795
I20260811 19:02:44.609409 28703 heartbeater.cc:464] Registering TS with master...
I20260811 19:02:44.610486 28703 heartbeater.cc:511] Master 127.29.112.190:35795 requested a full tablet report, sending...
I20260811 19:02:44.613569 28361 ts_manager.cc:194] Registered new tserver with Master: 6399a55532ef4ee382a1a3111bf92a85 (127.29.112.129:39721)
I20260811 19:02:44.616140 28361 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.129@KRBTEST.COM'} at 127.29.112.129:49911
WARNING: no policy specified for kudu/127.29.112.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.130.keytab.
Entry for principal kudu/127.29.112.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.130.keytab.
I20260811 19:02:44.681077 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.130:0
--local_ip_for_outbound_sockets=127.29.112.130
--webserver_interface=127.29.112.130
--webserver_port=0
--tserver_master_addrs=127.29.112.190:35795
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.130.keytab
--principal=kudu/127.29.112.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:35575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:02:45.130786 28709 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:02:45.131142 28709 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:02:45.131327 28709 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:02:45.138805 28709 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:02:45.138983 28709 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:02:45.139180 28709 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.130
I20260811 19:02:45.152406 28709 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:35575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.130.keytab
--principal=kudu/127.29.112.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.112.130
--webserver_port=0
--tserver_master_addrs=127.29.112.190:35795
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.130
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:02:45.154948 28709 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:02:45.156947 28709 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:02:45.173480 28730 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:02:45.173710 28722 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:02:45.173717 28725 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:02:45.175207 28709 server_base.cc:1034] running on GCE node
I20260811 19:02:45.176071 28709 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:02:45.177220 28709 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:02:45.178721 28709 hybrid_clock.cc:648] HybridClock initialized: now 1786474965178661 us; error 65 us; skew 500 ppm
I20260811 19:02:45.179329 28709 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:02:45 dist-test-slave-h6n0 krb5kdc[28278](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474965, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:02:45.184643 28709 init.cc:377] Logged in from keytab as kudu/127.29.112.130@KRBTEST.COM (short username kudu)
I20260811 19:02:45.187206 28709 webserver.cc:460] Webserver started at http://127.29.112.130:34853/ using document root <none> and password file <none>
I20260811 19:02:45.188252 28709 fs_manager.cc:359] Metadata directory not provided
I20260811 19:02:45.188442 28709 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:02:45.189263 28709 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:02:45.193907 28709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/minicluster-data/ts-1/data/instance:
uuid: "89d1759dc0a94e95a542ac06aa4888de"
format_stamp: "Formatted at 2026-08-11 19:02:45 on dist-test-slave-h6n0"
server_key: "0e68da166a4dd16d3e2499ca04e08f70"
server_key_iv: "9f2dfa8ca1253fa95e4689a01aba407a"
server_key_version: "encryptionkey@0"
I20260811 19:02:45.195008 28709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/minicluster-data/ts-1/wal/instance:
uuid: "89d1759dc0a94e95a542ac06aa4888de"
format_stamp: "Formatted at 2026-08-11 19:02:45 on dist-test-slave-h6n0"
server_key: "0e68da166a4dd16d3e2499ca04e08f70"
server_key_iv: "9f2dfa8ca1253fa95e4689a01aba407a"
server_key_version: "encryptionkey@0"
I20260811 19:02:45.201332 28709 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 19:02:45.206130 28741 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:45.208271 28709 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 19:02:45.208511 28709 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/minicluster-data/ts-1/wal
uuid: "89d1759dc0a94e95a542ac06aa4888de"
format_stamp: "Formatted at 2026-08-11 19:02:45 on dist-test-slave-h6n0"
server_key: "0e68da166a4dd16d3e2499ca04e08f70"
server_key_iv: "9f2dfa8ca1253fa95e4689a01aba407a"
server_key_version: "encryptionkey@0"
I20260811 19:02:45.208781 28709 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:02:45.227802 28709 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:02:45.233368 28709 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:02:45.233724 28709 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:02:45.234895 28709 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:02:45.236690 28709 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:02:45.236820 28709 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:45.236999 28709 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:02:45.237082 28709 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:45.282378 28709 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.130:44551
I20260811 19:02:45.282430 28930 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.130:44551 every 8 connection(s)
I20260811 19:02:45.284659 28709 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/minicluster-data/ts-1/data/info.pb
I20260811 19:02:45.288347 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 28709
I20260811 19:02:45.288587 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/minicluster-data/ts-1/wal/instance
I20260811 19:02:45.289203 30146 external_mini_cluster.cc:1428] Setting key 2442f03c4067fb47140eb3e02ecaa55a
Aug 11 19:02:45 dist-test-slave-h6n0 krb5kdc[28278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474965, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.130@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 19:02:45.318602 28935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:45.288245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.130:44343 (local address 127.29.112.190:35795)
0811 19:02:45.288956 (+ 711us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:45.288962 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:45.290202 (+ 1240us) server_negotiation.cc:407] Connection header received
0811 19:02:45.294063 (+ 3861us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:45.294070 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:45.294187 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:45.294326 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:02:45.296330 (+ 2004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:45.297158 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:45.299151 (+ 1993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:45.299641 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:45.299804 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:45.305704 (+ 5900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:02:45.305717 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:02:45.305726 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:02:45.305803 (+ 77us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:02:45.311546 (+ 5743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:45.313645 (+ 2099us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:45.313653 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:45.313656 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:45.314746 (+ 1090us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:45.315782 (+ 1036us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:45.315788 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:45.315790 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:45.316372 (+ 582us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:02:45.316663 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:45.317742 (+ 1079us) server_negotiation.cc:299] Negotiation successful
0811 19:02:45.318102 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":437,"thread_start_us":323,"threads_started":1}
I20260811 19:02:45.320149 28936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:45.288954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:35795 (local address 127.29.112.130:44343)
0811 19:02:45.289997 (+ 1043us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:45.290073 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0811 19:02:45.293702 (+ 3629us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:02:45.294717 (+ 1015us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:02:45.294730 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:02:45.295427 (+ 697us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:02:45.296101 (+ 674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:45.296122 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:45.297419 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:45.297425 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:45.298969 (+ 1544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:45.298985 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:45.299919 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:45.299927 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:45.300210 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:45.301859 (+ 1649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:02:45.301887 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:02:45.304982 (+ 3095us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:02:45.311928 (+ 6946us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:02:45.311943 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:02:45.311966 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:02:45.313420 (+ 1454us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:02:45.315065 (+ 1645us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:02:45.315076 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:02:45.315084 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:02:45.315541 (+ 457us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:02:45.316586 (+ 1045us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:02:45.316596 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:02:45.317209 (+ 613us) client_negotiation.cc:769] Sending connection context
0811 19:02:45.317592 (+ 383us) client_negotiation.cc:240] Negotiation successful
0811 19:02:45.318351 (+ 759us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":694,"thread_start_us":411,"threads_started":1}
I20260811 19:02:45.322817 28931 heartbeater.cc:347] Connected to a master server at 127.29.112.190:35795
I20260811 19:02:45.323297 28931 heartbeater.cc:464] Registering TS with master...
I20260811 19:02:45.324394 28931 heartbeater.cc:511] Master 127.29.112.190:35795 requested a full tablet report, sending...
I20260811 19:02:45.326781 28361 ts_manager.cc:194] Registered new tserver with Master: 89d1759dc0a94e95a542ac06aa4888de (127.29.112.130:44551)
I20260811 19:02:45.327795 28361 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.130@KRBTEST.COM'} at 127.29.112.130:44343
WARNING: no policy specified for kudu/127.29.112.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.131.keytab.
Entry for principal kudu/127.29.112.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.131.keytab.
W20260811 19:02:45.396699 30146 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.29.112.131: real 0.103s user 0.000s sys 0.038s
I20260811 19:02:45.398021 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.131:0
--local_ip_for_outbound_sockets=127.29.112.131
--webserver_interface=127.29.112.131
--webserver_port=0
--tserver_master_addrs=127.29.112.190:35795
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.131.keytab
--principal=kudu/127.29.112.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:35575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 19:02:45.620965 28703 heartbeater.cc:503] Master 127.29.112.190:35795 was elected leader, sending a full tablet report...
W20260811 19:02:45.871130 28943 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:02:45.871536 28943 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:02:45.871729 28943 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:02:45.881569 28943 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:02:45.881745 28943 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:02:45.882007 28943 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.131
I20260811 19:02:45.893581 28943 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:35575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.131.keytab
--principal=kudu/127.29.112.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.112.131
--webserver_port=0
--tserver_master_addrs=127.29.112.190:35795
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.131
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:02:45.895145 28943 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:02:45.896588 28943 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:02:45.913303 28950 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:02:45.913353 28956 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:02:45.913353 28954 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:02:45.915623 28943 server_base.cc:1034] running on GCE node
I20260811 19:02:45.916471 28943 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:02:45.918354 28943 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:02:45.919831 28943 hybrid_clock.cc:648] HybridClock initialized: now 1786474965919740 us; error 87 us; skew 500 ppm
I20260811 19:02:45.920352 28943 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:02:45 dist-test-slave-h6n0 krb5kdc[28278](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474965, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:02:45.924955 28943 init.cc:377] Logged in from keytab as kudu/127.29.112.131@KRBTEST.COM (short username kudu)
I20260811 19:02:45.926888 28943 webserver.cc:460] Webserver started at http://127.29.112.131:36747/ using document root <none> and password file <none>
I20260811 19:02:45.927724 28943 fs_manager.cc:359] Metadata directory not provided
I20260811 19:02:45.927850 28943 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:02:45.928220 28943 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:02:45.931753 28943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/minicluster-data/ts-2/data/instance:
uuid: "20cfeed44e8041cbbbedae11c0e3ba07"
format_stamp: "Formatted at 2026-08-11 19:02:45 on dist-test-slave-h6n0"
server_key: "11e3e936df5e49309014645d5a1ef2e3"
server_key_iv: "8dae2d500f735dc879ca434b06d7a8dc"
server_key_version: "encryptionkey@0"
I20260811 19:02:45.932705 28943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/minicluster-data/ts-2/wal/instance:
uuid: "20cfeed44e8041cbbbedae11c0e3ba07"
format_stamp: "Formatted at 2026-08-11 19:02:45 on dist-test-slave-h6n0"
server_key: "11e3e936df5e49309014645d5a1ef2e3"
server_key_iv: "8dae2d500f735dc879ca434b06d7a8dc"
server_key_version: "encryptionkey@0"
I20260811 19:02:45.943109 28943 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.010s sys 0.001s
I20260811 19:02:45.947587 28970 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:45.950647 28943 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.006s sys 0.000s
I20260811 19:02:45.950961 28943 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/minicluster-data/ts-2/wal
uuid: "20cfeed44e8041cbbbedae11c0e3ba07"
format_stamp: "Formatted at 2026-08-11 19:02:45 on dist-test-slave-h6n0"
server_key: "11e3e936df5e49309014645d5a1ef2e3"
server_key_iv: "8dae2d500f735dc879ca434b06d7a8dc"
server_key_version: "encryptionkey@0"
I20260811 19:02:45.951242 28943 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:02:45.975600 28943 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:02:45.984139 28943 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:02:45.984688 28943 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:02:45.986025 28943 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:02:45.987792 28943 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:02:45.987890 28943 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:45.988040 28943 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:02:45.988121 28943 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:46.046157 28943 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.131:36637
I20260811 19:02:46.046517 29157 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.131:36637 every 8 connection(s)
I20260811 19:02:46.049120 28943 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/minicluster-data/ts-2/data/info.pb
I20260811 19:02:46.055478 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 28943
I20260811 19:02:46.055765 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/minicluster-data/ts-2/wal/instance
I20260811 19:02:46.056654 30146 external_mini_cluster.cc:1428] Setting key 3bc9c31cf574631aba3e4e777034d8c9
Aug 11 19:02:46 dist-test-slave-h6n0 krb5kdc[28278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474965, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.131@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 19:02:46.089484 29163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:46.056118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.131:40597 (local address 127.29.112.190:35795)
0811 19:02:46.057992 (+ 1874us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:46.058023 (+ 31us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:46.059360 (+ 1337us) server_negotiation.cc:407] Connection header received
0811 19:02:46.062897 (+ 3537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:46.062903 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:46.062993 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:46.063146 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:02:46.065975 (+ 2829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:46.067000 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:46.069441 (+ 2441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:46.069901 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:46.070019 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:46.078071 (+ 8052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:02:46.078077 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:02:46.078082 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:02:46.078231 (+ 149us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:02:46.083641 (+ 5410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:46.084732 (+ 1091us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:46.084739 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:46.084742 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:46.084989 (+ 247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:46.086073 (+ 1084us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:46.086078 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:46.086080 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:46.087294 (+ 1214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:02:46.087486 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:46.088743 (+ 1257us) server_negotiation.cc:299] Negotiation successful
0811 19:02:46.089035 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1542,"thread_start_us":1387,"threads_started":1}
I20260811 19:02:46.090917 29164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:46.057034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:35795 (local address 127.29.112.131:40597)
0811 19:02:46.059032 (+ 1998us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:46.059101 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0811 19:02:46.062015 (+ 2914us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:02:46.063486 (+ 1471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:02:46.063530 (+ 44us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:02:46.064507 (+ 977us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:02:46.065459 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:46.065488 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:46.067382 (+ 1894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:46.067388 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:46.069182 (+ 1794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:46.069204 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:46.070187 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:46.070194 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:46.070779 (+ 585us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:46.072297 (+ 1518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:02:46.072326 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:02:46.077426 (+ 5100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:02:46.083857 (+ 6431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:02:46.083871 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:02:46.083892 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:02:46.084505 (+ 613us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:02:46.085373 (+ 868us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:02:46.085378 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:02:46.085381 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:02:46.085707 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:02:46.087676 (+ 1969us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:02:46.087686 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:02:46.088282 (+ 596us) client_negotiation.cc:769] Sending connection context
0811 19:02:46.088646 (+ 364us) client_negotiation.cc:240] Negotiation successful
0811 19:02:46.089291 (+ 645us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1658,"thread_start_us":992,"threads_started":1}
I20260811 19:02:46.094307 29158 heartbeater.cc:347] Connected to a master server at 127.29.112.190:35795
I20260811 19:02:46.094796 29158 heartbeater.cc:464] Registering TS with master...
I20260811 19:02:46.096681 29158 heartbeater.cc:511] Master 127.29.112.190:35795 requested a full tablet report, sending...
I20260811 19:02:46.099565 28361 ts_manager.cc:194] Registered new tserver with Master: 20cfeed44e8041cbbbedae11c0e3ba07 (127.29.112.131:36637)
I20260811 19:02:46.101255 28361 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.131@KRBTEST.COM'} at 127.29.112.131:40597
I20260811 19:02:46.106714 30146 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 19:02:46.135445 29163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:46.112448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39054 (local address 127.29.112.190:35795)
0811 19:02:46.112660 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:46.112666 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:46.113908 (+ 1242us) server_negotiation.cc:407] Connection header received
0811 19:02:46.114473 (+ 565us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:46.114478 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:46.114632 (+ 154us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:46.114737 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:02:46.117605 (+ 2868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:46.118771 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:46.121179 (+ 2408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:46.121862 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:46.122006 (+ 144us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:46.126223 (+ 4217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:02:46.126230 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:02:46.126234 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:02:46.126280 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:02:46.130165 (+ 3885us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:46.131272 (+ 1107us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:46.131276 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:46.131278 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:46.131440 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:46.132133 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:46.132138 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:46.132140 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:46.132555 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:02:46.133971 (+ 1416us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:46.134320 (+ 349us) server_negotiation.cc:299] Negotiation successful
0811 19:02:46.134857 (+ 537us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 19:02:46.143249 28361 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39054:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 19:02:46.151554 28361 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 19:02:46.189004 29189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:46.171082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48764 (local address 127.29.112.130:44551)
0811 19:02:46.172943 (+ 1861us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:46.172957 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:46.174403 (+ 1446us) server_negotiation.cc:407] Connection header received
0811 19:02:46.174646 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:46.174653 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:46.174892 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:46.175735 (+ 843us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:02:46.178881 (+ 3146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:46.181284 (+ 2403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:46.185677 (+ 4393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:46.187047 (+ 1370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:46.187260 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:46.187491 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:46.187860 (+ 369us) server_negotiation.cc:299] Negotiation successful
0811 19:02:46.188620 (+ 760us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1630,"thread_start_us":1477,"threads_started":1}
I20260811 19:02:46.189776 29188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:46.170680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60356 (local address 127.29.112.131:36637)
0811 19:02:46.174079 (+ 3399us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:46.174091 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:46.174154 (+ 63us) server_negotiation.cc:407] Connection header received
0811 19:02:46.174339 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:46.174352 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:46.174763 (+ 411us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:46.175165 (+ 402us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:02:46.177839 (+ 2674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:46.180833 (+ 2994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:46.185716 (+ 4883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:46.187494 (+ 1778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:46.187741 (+ 247us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:46.187948 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:46.188795 (+ 847us) server_negotiation.cc:299] Negotiation successful
0811 19:02:46.189097 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3062,"thread_start_us":2752,"threads_started":1}
I20260811 19:02:46.190277 29187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:46.170556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.130:44551 (local address 127.0.0.1:48764)
0811 19:02:46.174196 (+ 3640us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:46.174218 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 19:02:46.174415 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:02:46.176058 (+ 1643us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:02:46.176068 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:02:46.176119 (+ 51us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:02:46.177500 (+ 1381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:46.177517 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:46.181462 (+ 3945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:46.181467 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:46.185445 (+ 3978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:46.185457 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:46.187285 (+ 1828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:46.187293 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:46.187523 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:46.187585 (+ 62us) client_negotiation.cc:769] Sending connection context
0811 19:02:46.189368 (+ 1783us) client_negotiation.cc:240] Negotiation successful
0811 19:02:46.189589 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3476,"mutex_wait_us":271,"thread_start_us":3304,"threads_started":1}
I20260811 19:02:46.190373 29191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:46.175378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:39721 (local address 127.0.0.1:42776)
0811 19:02:46.177649 (+ 2271us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:46.177677 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0811 19:02:46.177914 (+ 237us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:02:46.179414 (+ 1500us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:02:46.179419 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:02:46.179449 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:02:46.179876 (+ 427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:46.179892 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:46.182407 (+ 2515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:46.182418 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:46.185783 (+ 3365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:46.185800 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:46.187928 (+ 2128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:46.187935 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:46.188652 (+ 717us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:46.188707 (+ 55us) client_negotiation.cc:769] Sending connection context
0811 19:02:46.188965 (+ 258us) client_negotiation.cc:240] Negotiation successful
0811 19:02:46.189511 (+ 546us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2036,"mutex_wait_us":6,"thread_start_us":1324,"threads_started":1}
I20260811 19:02:46.191330 29186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:46.169955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.131:36637 (local address 127.0.0.1:60356)
0811 19:02:46.172104 (+ 2149us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:46.172160 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 19:02:46.172395 (+ 235us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:02:46.175361 (+ 2966us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:02:46.175376 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:02:46.175428 (+ 52us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:02:46.177423 (+ 1995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:46.177454 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:46.181372 (+ 3918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:46.181383 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:46.185437 (+ 4054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:46.185457 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:46.187726 (+ 2269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:46.187736 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:46.188389 (+ 653us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:46.188445 (+ 56us) client_negotiation.cc:769] Sending connection context
0811 19:02:46.188641 (+ 196us) client_negotiation.cc:240] Negotiation successful
0811 19:02:46.188877 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1771,"mutex_wait_us":390,"thread_start_us":1563,"threads_started":1}
I20260811 19:02:46.192030 29193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:46.175801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42776 (local address 127.29.112.129:39721)
0811 19:02:46.178617 (+ 2816us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:46.178628 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:46.178663 (+ 35us) server_negotiation.cc:407] Connection header received
0811 19:02:46.178798 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:46.178809 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:46.179142 (+ 333us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:46.180100 (+ 958us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:02:46.180399 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:46.182002 (+ 1603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:46.185998 (+ 3996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:46.187657 (+ 1659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:46.191201 (+ 3544us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:46.191408 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:46.191498 (+ 90us) server_negotiation.cc:299] Negotiation successful
0811 19:02:46.191685 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2602,"thread_start_us":2417,"threads_started":1}
I20260811 19:02:46.195952 29051 tablet_service.cc:1467] Processing CreateTablet for tablet 241b075a469f4082a2085a7897a33577 (DEFAULT_TABLE table=test-table [id=e92593f13b56490c80662157df8bf4a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:02:46.195771 28826 tablet_service.cc:1467] Processing CreateTablet for tablet 241b075a469f4082a2085a7897a33577 (DEFAULT_TABLE table=test-table [id=e92593f13b56490c80662157df8bf4a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:02:46.196979 28585 tablet_service.cc:1467] Processing CreateTablet for tablet 241b075a469f4082a2085a7897a33577 (DEFAULT_TABLE table=test-table [id=e92593f13b56490c80662157df8bf4a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:02:46.198509 29051 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 241b075a469f4082a2085a7897a33577. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:02:46.198704 28826 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 241b075a469f4082a2085a7897a33577. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:02:46.200160 28585 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 241b075a469f4082a2085a7897a33577. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:02:46.213049 29195 tablet_bootstrap.cc:492] T 241b075a469f4082a2085a7897a33577 P 20cfeed44e8041cbbbedae11c0e3ba07: Bootstrap starting.
I20260811 19:02:46.217931 29196 tablet_bootstrap.cc:492] T 241b075a469f4082a2085a7897a33577 P 89d1759dc0a94e95a542ac06aa4888de: Bootstrap starting.
I20260811 19:02:46.222805 29195 tablet_bootstrap.cc:654] T 241b075a469f4082a2085a7897a33577 P 20cfeed44e8041cbbbedae11c0e3ba07: Neither blocks nor log segments found. Creating new log.
I20260811 19:02:46.223500 29198 tablet_bootstrap.cc:492] T 241b075a469f4082a2085a7897a33577 P 6399a55532ef4ee382a1a3111bf92a85: Bootstrap starting.
I20260811 19:02:46.225294 29196 tablet_bootstrap.cc:654] T 241b075a469f4082a2085a7897a33577 P 89d1759dc0a94e95a542ac06aa4888de: Neither blocks nor log segments found. Creating new log.
I20260811 19:02:46.225246 29195 log.cc:824] T 241b075a469f4082a2085a7897a33577 P 20cfeed44e8041cbbbedae11c0e3ba07: Log is configured to *not* fsync() on all Append() calls
I20260811 19:02:46.226831 29196 log.cc:824] T 241b075a469f4082a2085a7897a33577 P 89d1759dc0a94e95a542ac06aa4888de: Log is configured to *not* fsync() on all Append() calls
I20260811 19:02:46.227409 29198 tablet_bootstrap.cc:654] T 241b075a469f4082a2085a7897a33577 P 6399a55532ef4ee382a1a3111bf92a85: Neither blocks nor log segments found. Creating new log.
I20260811 19:02:46.228547 29198 log.cc:824] T 241b075a469f4082a2085a7897a33577 P 6399a55532ef4ee382a1a3111bf92a85: Log is configured to *not* fsync() on all Append() calls
I20260811 19:02:46.231725 29198 tablet_bootstrap.cc:492] T 241b075a469f4082a2085a7897a33577 P 6399a55532ef4ee382a1a3111bf92a85: No bootstrap required, opened a new log
I20260811 19:02:46.232097 29198 ts_tablet_manager.cc:1397] T 241b075a469f4082a2085a7897a33577 P 6399a55532ef4ee382a1a3111bf92a85: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260811 19:02:46.234654 29195 tablet_bootstrap.cc:492] T 241b075a469f4082a2085a7897a33577 P 20cfeed44e8041cbbbedae11c0e3ba07: No bootstrap required, opened a new log
I20260811 19:02:46.234977 29195 ts_tablet_manager.cc:1397] T 241b075a469f4082a2085a7897a33577 P 20cfeed44e8041cbbbedae11c0e3ba07: Time spent bootstrapping tablet: real 0.022s user 0.007s sys 0.011s
I20260811 19:02:46.235386 29196 tablet_bootstrap.cc:492] T 241b075a469f4082a2085a7897a33577 P 89d1759dc0a94e95a542ac06aa4888de: No bootstrap required, opened a new log
I20260811 19:02:46.235678 29196 ts_tablet_manager.cc:1397] T 241b075a469f4082a2085a7897a33577 P 89d1759dc0a94e95a542ac06aa4888de: Time spent bootstrapping tablet: real 0.018s user 0.009s sys 0.008s
I20260811 19:02:46.245596 29198 raft_consensus.cc:348] T 241b075a469f4082a2085a7897a33577 P 6399a55532ef4ee382a1a3111bf92a85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20cfeed44e8041cbbbedae11c0e3ba07" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 36637 } } peers { permanent_uuid: "89d1759dc0a94e95a542ac06aa4888de" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 44551 } } peers { permanent_uuid: "6399a55532ef4ee382a1a3111bf92a85" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 39721 } }
I20260811 19:02:46.246199 29198 raft_consensus.cc:374] T 241b075a469f4082a2085a7897a33577 P 6399a55532ef4ee382a1a3111bf92a85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:02:46.246313 29198 raft_consensus.cc:729] T 241b075a469f4082a2085a7897a33577 P 6399a55532ef4ee382a1a3111bf92a85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6399a55532ef4ee382a1a3111bf92a85, State: Initialized, Role: FOLLOWER
I20260811 19:02:46.247154 29198 consensus_queue.cc:260] T 241b075a469f4082a2085a7897a33577 P 6399a55532ef4ee382a1a3111bf92a85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20cfeed44e8041cbbbedae11c0e3ba07" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 36637 } } peers { permanent_uuid: "89d1759dc0a94e95a542ac06aa4888de" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 44551 } } peers { permanent_uuid: "6399a55532ef4ee382a1a3111bf92a85" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 39721 } }
I20260811 19:02:46.247159 29195 raft_consensus.cc:348] T 241b075a469f4082a2085a7897a33577 P 20cfeed44e8041cbbbedae11c0e3ba07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20cfeed44e8041cbbbedae11c0e3ba07" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 36637 } } peers { permanent_uuid: "89d1759dc0a94e95a542ac06aa4888de" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 44551 } } peers { permanent_uuid: "6399a55532ef4ee382a1a3111bf92a85" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 39721 } }
I20260811 19:02:46.247581 29195 raft_consensus.cc:374] T 241b075a469f4082a2085a7897a33577 P 20cfeed44e8041cbbbedae11c0e3ba07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:02:46.247684 29195 raft_consensus.cc:729] T 241b075a469f4082a2085a7897a33577 P 20cfeed44e8041cbbbedae11c0e3ba07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20cfeed44e8041cbbbedae11c0e3ba07, State: Initialized, Role: FOLLOWER
I20260811 19:02:46.248423 29195 consensus_queue.cc:260] T 241b075a469f4082a2085a7897a33577 P 20cfeed44e8041cbbbedae11c0e3ba07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20cfeed44e8041cbbbedae11c0e3ba07" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 36637 } } peers { permanent_uuid: "89d1759dc0a94e95a542ac06aa4888de" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 44551 } } peers { permanent_uuid: "6399a55532ef4ee382a1a3111bf92a85" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 39721 } }
I20260811 19:02:46.248698 29196 raft_consensus.cc:348] T 241b075a469f4082a2085a7897a33577 P 89d1759dc0a94e95a542ac06aa4888de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20cfeed44e8041cbbbedae11c0e3ba07" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 36637 } } peers { permanent_uuid: "89d1759dc0a94e95a542ac06aa4888de" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 44551 } } peers { permanent_uuid: "6399a55532ef4ee382a1a3111bf92a85" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 39721 } }
I20260811 19:02:46.249337 29196 raft_consensus.cc:374] T 241b075a469f4082a2085a7897a33577 P 89d1759dc0a94e95a542ac06aa4888de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:02:46.249460 29196 raft_consensus.cc:729] T 241b075a469f4082a2085a7897a33577 P 89d1759dc0a94e95a542ac06aa4888de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89d1759dc0a94e95a542ac06aa4888de, State: Initialized, Role: FOLLOWER
I20260811 19:02:46.250080 29198 ts_tablet_manager.cc:1428] T 241b075a469f4082a2085a7897a33577 P 6399a55532ef4ee382a1a3111bf92a85: Time spent starting tablet: real 0.018s user 0.015s sys 0.004s
I20260811 19:02:46.250298 29196 consensus_queue.cc:260] T 241b075a469f4082a2085a7897a33577 P 89d1759dc0a94e95a542ac06aa4888de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20cfeed44e8041cbbbedae11c0e3ba07" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 36637 } } peers { permanent_uuid: "89d1759dc0a94e95a542ac06aa4888de" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 44551 } } peers { permanent_uuid: "6399a55532ef4ee382a1a3111bf92a85" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 39721 } }
I20260811 19:02:46.252588 29195 ts_tablet_manager.cc:1428] T 241b075a469f4082a2085a7897a33577 P 20cfeed44e8041cbbbedae11c0e3ba07: Time spent starting tablet: real 0.017s user 0.011s sys 0.005s
I20260811 19:02:46.254992 29158 heartbeater.cc:503] Master 127.29.112.190:35795 was elected leader, sending a full tablet report...
I20260811 19:02:46.255815 28931 heartbeater.cc:503] Master 127.29.112.190:35795 was elected leader, sending a full tablet report...
I20260811 19:02:46.256538 29196 ts_tablet_manager.cc:1428] T 241b075a469f4082a2085a7897a33577 P 89d1759dc0a94e95a542ac06aa4888de: Time spent starting tablet: real 0.021s user 0.010s sys 0.011s
W20260811 19:02:46.288715 28933 tablet.cc:2369] T 241b075a469f4082a2085a7897a33577 P 89d1759dc0a94e95a542ac06aa4888de: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 19:02:46.302804 29161 tablet.cc:2369] T 241b075a469f4082a2085a7897a33577 P 20cfeed44e8041cbbbedae11c0e3ba07: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 19:02:46.330122 28704 tablet.cc:2369] T 241b075a469f4082a2085a7897a33577 P 6399a55532ef4ee382a1a3111bf92a85: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 19:02:46.337486 29204 raft_consensus.cc:482] T 241b075a469f4082a2085a7897a33577 P 6399a55532ef4ee382a1a3111bf92a85 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:02:46.337839 29204 raft_consensus.cc:504] T 241b075a469f4082a2085a7897a33577 P 6399a55532ef4ee382a1a3111bf92a85 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20cfeed44e8041cbbbedae11c0e3ba07" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 36637 } } peers { permanent_uuid: "89d1759dc0a94e95a542ac06aa4888de" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 44551 } } peers { permanent_uuid: "6399a55532ef4ee382a1a3111bf92a85" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 39721 } }
I20260811 19:02:46.340907 29204 leader_election.cc:288] T 241b075a469f4082a2085a7897a33577 P 6399a55532ef4ee382a1a3111bf92a85 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 20cfeed44e8041cbbbedae11c0e3ba07 (127.29.112.131:36637), 89d1759dc0a94e95a542ac06aa4888de (127.29.112.130:44551)
I20260811 19:02:46.351014 29188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:46.341390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.129:39677 (local address 127.29.112.131:36637)
0811 19:02:46.341687 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:46.341694 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:46.343001 (+ 1307us) server_negotiation.cc:407] Connection header received
0811 19:02:46.343217 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:46.343227 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:46.343320 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:46.343440 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:02:46.344618 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:46.345505 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:46.348438 (+ 2933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:46.349531 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:46.349679 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:46.349740 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:46.350355 (+ 615us) server_negotiation.cc:299] Negotiation successful
0811 19:02:46.350485 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260811 19:02:46.351094 29209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:46.341684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.131:36637 (local address 127.29.112.129:39677)
0811 19:02:46.342853 (+ 1169us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:46.342879 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 19:02:46.343055 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:02:46.343564 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:02:46.343569 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:02:46.343606 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:02:46.344377 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:46.344406 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:46.345694 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:46.345702 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:46.348131 (+ 2429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:46.348151 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:46.349797 (+ 1646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:46.349804 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:46.350045 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:46.350088 (+ 43us) client_negotiation.cc:769] Sending connection context
0811 19:02:46.350275 (+ 187us) client_negotiation.cc:240] Negotiation successful
0811 19:02:46.350466 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":773,"mutex_wait_us":8,"thread_start_us":491,"threads_started":1}
I20260811 19:02:46.352034 29210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:46.341749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.130:44551 (local address 127.29.112.129:41645)
0811 19:02:46.343144 (+ 1395us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:46.343171 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 19:02:46.343319 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:02:46.343876 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:02:46.343880 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:02:46.343902 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:02:46.344460 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:46.344475 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:46.346428 (+ 1953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:46.346438 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:46.348774 (+ 2336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:46.348793 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:46.351312 (+ 2519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:46.351325 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:46.351528 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:46.351567 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 19:02:46.351669 (+ 102us) client_negotiation.cc:240] Negotiation successful
0811 19:02:46.351786 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1226,"thread_start_us":961,"threads_started":1}
I20260811 19:02:46.352595 29189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:46.341895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.129:41645 (local address 127.29.112.130:44551)
0811 19:02:46.342191 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:46.342200 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:46.343320 (+ 1120us) server_negotiation.cc:407] Connection header received
0811 19:02:46.343608 (+ 288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:46.343615 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:46.343706 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:46.343835 (+ 129us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:02:46.344859 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:46.346190 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:46.349041 (+ 2851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:46.350691 (+ 1650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:46.350884 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:46.350964 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:46.351713 (+ 749us) server_negotiation.cc:299] Negotiation successful
0811 19:02:46.352028 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260811 19:02:46.352682 29088 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "241b075a469f4082a2085a7897a33577" candidate_uuid: "6399a55532ef4ee382a1a3111bf92a85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20cfeed44e8041cbbbedae11c0e3ba07" is_pre_election: true
I20260811 19:02:46.353308 29088 raft_consensus.cc:2454] T 241b075a469f4082a2085a7897a33577 P 20cfeed44e8041cbbbedae11c0e3ba07 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6399a55532ef4ee382a1a3111bf92a85 in term 0.
I20260811 19:02:46.353555 28856 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "241b075a469f4082a2085a7897a33577" candidate_uuid: "6399a55532ef4ee382a1a3111bf92a85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89d1759dc0a94e95a542ac06aa4888de" is_pre_election: true
I20260811 19:02:46.354022 28856 raft_consensus.cc:2454] T 241b075a469f4082a2085a7897a33577 P 89d1759dc0a94e95a542ac06aa4888de [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6399a55532ef4ee382a1a3111bf92a85 in term 0.
I20260811 19:02:46.354054 28499 leader_election.cc:302] T 241b075a469f4082a2085a7897a33577 P 6399a55532ef4ee382a1a3111bf92a85 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20cfeed44e8041cbbbedae11c0e3ba07, 6399a55532ef4ee382a1a3111bf92a85; no voters:
I20260811 19:02:46.354591 29204 raft_consensus.cc:2781] T 241b075a469f4082a2085a7897a33577 P 6399a55532ef4ee382a1a3111bf92a85 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 19:02:46.354743 29204 raft_consensus.cc:482] T 241b075a469f4082a2085a7897a33577 P 6399a55532ef4ee382a1a3111bf92a85 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 19:02:46.354866 29204 raft_consensus.cc:3037] T 241b075a469f4082a2085a7897a33577 P 6399a55532ef4ee382a1a3111bf92a85 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:02:46.358198 29204 raft_consensus.cc:504] T 241b075a469f4082a2085a7897a33577 P 6399a55532ef4ee382a1a3111bf92a85 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20cfeed44e8041cbbbedae11c0e3ba07" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 36637 } } peers { permanent_uuid: "89d1759dc0a94e95a542ac06aa4888de" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 44551 } } peers { permanent_uuid: "6399a55532ef4ee382a1a3111bf92a85" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 39721 } }
I20260811 19:02:46.359026 29204 leader_election.cc:288] T 241b075a469f4082a2085a7897a33577 P 6399a55532ef4ee382a1a3111bf92a85 [CANDIDATE]: Term 1 election: Requested vote from peers 20cfeed44e8041cbbbedae11c0e3ba07 (127.29.112.131:36637), 89d1759dc0a94e95a542ac06aa4888de (127.29.112.130:44551)
I20260811 19:02:46.359814 29088 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "241b075a469f4082a2085a7897a33577" candidate_uuid: "6399a55532ef4ee382a1a3111bf92a85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20cfeed44e8041cbbbedae11c0e3ba07"
I20260811 19:02:46.360121 29088 raft_consensus.cc:3037] T 241b075a469f4082a2085a7897a33577 P 20cfeed44e8041cbbbedae11c0e3ba07 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:02:46.360242 28856 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "241b075a469f4082a2085a7897a33577" candidate_uuid: "6399a55532ef4ee382a1a3111bf92a85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89d1759dc0a94e95a542ac06aa4888de"
I20260811 19:02:46.360615 28856 raft_consensus.cc:3037] T 241b075a469f4082a2085a7897a33577 P 89d1759dc0a94e95a542ac06aa4888de [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:02:46.363193 29088 raft_consensus.cc:2454] T 241b075a469f4082a2085a7897a33577 P 20cfeed44e8041cbbbedae11c0e3ba07 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6399a55532ef4ee382a1a3111bf92a85 in term 1.
I20260811 19:02:46.363777 28499 leader_election.cc:302] T 241b075a469f4082a2085a7897a33577 P 6399a55532ef4ee382a1a3111bf92a85 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20cfeed44e8041cbbbedae11c0e3ba07, 6399a55532ef4ee382a1a3111bf92a85; no voters:
I20260811 19:02:46.364114 29204 raft_consensus.cc:2781] T 241b075a469f4082a2085a7897a33577 P 6399a55532ef4ee382a1a3111bf92a85 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:02:46.364928 29204 raft_consensus.cc:686] T 241b075a469f4082a2085a7897a33577 P 6399a55532ef4ee382a1a3111bf92a85 [term 1 LEADER]: Becoming Leader. State: Replica: 6399a55532ef4ee382a1a3111bf92a85, State: Running, Role: LEADER
I20260811 19:02:46.365362 28856 raft_consensus.cc:2454] T 241b075a469f4082a2085a7897a33577 P 89d1759dc0a94e95a542ac06aa4888de [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6399a55532ef4ee382a1a3111bf92a85 in term 1.
I20260811 19:02:46.365588 29204 consensus_queue.cc:237] T 241b075a469f4082a2085a7897a33577 P 6399a55532ef4ee382a1a3111bf92a85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20cfeed44e8041cbbbedae11c0e3ba07" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 36637 } } peers { permanent_uuid: "89d1759dc0a94e95a542ac06aa4888de" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 44551 } } peers { permanent_uuid: "6399a55532ef4ee382a1a3111bf92a85" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 39721 } }
I20260811 19:02:46.372269 28358 catalog_manager.cc:5495] T 241b075a469f4082a2085a7897a33577 P 6399a55532ef4ee382a1a3111bf92a85 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6399a55532ef4ee382a1a3111bf92a85 (127.29.112.129). New cstate: current_term: 1 leader_uuid: "6399a55532ef4ee382a1a3111bf92a85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20cfeed44e8041cbbbedae11c0e3ba07" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 36637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "89d1759dc0a94e95a542ac06aa4888de" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 44551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6399a55532ef4ee382a1a3111bf92a85" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 39721 } health_report { overall_health: HEALTHY } } }
I20260811 19:02:46.441037 29193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:46.430893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42780 (local address 127.29.112.129:39721)
0811 19:02:46.431274 (+ 381us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:46.431280 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:46.431624 (+ 344us) server_negotiation.cc:407] Connection header received
0811 19:02:46.431956 (+ 332us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:46.431962 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:46.432079 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:46.432409 (+ 330us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:02:46.432926 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:46.433967 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:46.436194 (+ 2227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:46.437282 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:46.437437 (+ 155us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:46.438762 (+ 1325us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:02:46.439823 (+ 1061us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:02:46.440045 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:46.440515 (+ 470us) server_negotiation.cc:299] Negotiation successful
0811 19:02:46.440599 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":234}
I20260811 19:02:46.452811 28856 raft_consensus.cc:1261] T 241b075a469f4082a2085a7897a33577 P 89d1759dc0a94e95a542ac06aa4888de [term 1 FOLLOWER]: Refusing update from remote peer 6399a55532ef4ee382a1a3111bf92a85: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:02:46.452941 29088 raft_consensus.cc:1261] T 241b075a469f4082a2085a7897a33577 P 20cfeed44e8041cbbbedae11c0e3ba07 [term 1 FOLLOWER]: Refusing update from remote peer 6399a55532ef4ee382a1a3111bf92a85: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:02:46.454682 29212 consensus_queue.cc:1035] T 241b075a469f4082a2085a7897a33577 P 6399a55532ef4ee382a1a3111bf92a85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20cfeed44e8041cbbbedae11c0e3ba07" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 36637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:02:46.455374 29204 consensus_queue.cc:1035] T 241b075a469f4082a2085a7897a33577 P 6399a55532ef4ee382a1a3111bf92a85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "89d1759dc0a94e95a542ac06aa4888de" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 44551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:02:46.479671 29221 mvcc.cc:205] Tried to move back new op lower bound from 7317401462575476736 to 7317401462240370688. Current Snapshot: MvccSnapshot[applied={T|T < 7317401462575476736}]
I20260811 19:02:46.481729 29219 mvcc.cc:205] Tried to move back new op lower bound from 7317401462575476736 to 7317401462240370688. Current Snapshot: MvccSnapshot[applied={T|T < 7317401462575476736}]
I20260811 19:02:46.498198 28358 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39054:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:02:46.498634 28358 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39054:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:02:46.505431 28358 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fb4a48b2a3d148129d73bf9f06451002: Sending DeleteTablet for 3 replicas of tablet 241b075a469f4082a2085a7897a33577
I20260811 19:02:46.507153 29051 tablet_service.cc:1514] Processing DeleteTablet for tablet 241b075a469f4082a2085a7897a33577 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:02:46 UTC) from {username='kudu', principal='kudu/127.29.112.190@KRBTEST.COM'} at 127.0.0.1:60356
I20260811 19:02:46.507153 28826 tablet_service.cc:1514] Processing DeleteTablet for tablet 241b075a469f4082a2085a7897a33577 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:02:46 UTC) from {username='kudu', principal='kudu/127.29.112.190@KRBTEST.COM'} at 127.0.0.1:48764
I20260811 19:02:46.508795 28585 tablet_service.cc:1514] Processing DeleteTablet for tablet 241b075a469f4082a2085a7897a33577 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:02:46 UTC) from {username='kudu', principal='kudu/127.29.112.190@KRBTEST.COM'} at 127.0.0.1:42776
I20260811 19:02:46.510445 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 28470
I20260811 19:02:46.518786 29235 tablet_replica.cc:330] T 241b075a469f4082a2085a7897a33577 P 89d1759dc0a94e95a542ac06aa4888de: stopping tablet replica
I20260811 19:02:46.518867 29234 tablet_replica.cc:330] T 241b075a469f4082a2085a7897a33577 P 20cfeed44e8041cbbbedae11c0e3ba07: stopping tablet replica
I20260811 19:02:46.519480 29235 raft_consensus.cc:2229] T 241b075a469f4082a2085a7897a33577 P 89d1759dc0a94e95a542ac06aa4888de [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:02:46.519488 29234 raft_consensus.cc:2229] T 241b075a469f4082a2085a7897a33577 P 20cfeed44e8041cbbbedae11c0e3ba07 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:02:46.520169 29235 raft_consensus.cc:2258] T 241b075a469f4082a2085a7897a33577 P 89d1759dc0a94e95a542ac06aa4888de [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:02:46.520190 29234 raft_consensus.cc:2258] T 241b075a469f4082a2085a7897a33577 P 20cfeed44e8041cbbbedae11c0e3ba07 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:02:46.522879 29235 ts_tablet_manager.cc:1905] T 241b075a469f4082a2085a7897a33577 P 89d1759dc0a94e95a542ac06aa4888de: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:02:46.522905 29234 ts_tablet_manager.cc:1905] T 241b075a469f4082a2085a7897a33577 P 20cfeed44e8041cbbbedae11c0e3ba07: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:02:46.530797 29235 ts_tablet_manager.cc:1918] T 241b075a469f4082a2085a7897a33577 P 89d1759dc0a94e95a542ac06aa4888de: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:02:46.531224 29235 log.cc:1196] T 241b075a469f4082a2085a7897a33577 P 89d1759dc0a94e95a542ac06aa4888de: Deleting WAL directory at /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/minicluster-data/ts-1/wal/wals/241b075a469f4082a2085a7897a33577
I20260811 19:02:46.531575 29239 tablet_replica.cc:330] T 241b075a469f4082a2085a7897a33577 P 6399a55532ef4ee382a1a3111bf92a85: stopping tablet replica
I20260811 19:02:46.532533 29235 ts_tablet_manager.cc:1939] T 241b075a469f4082a2085a7897a33577 P 89d1759dc0a94e95a542ac06aa4888de: Deleting consensus metadata
I20260811 19:02:46.532405 29239 raft_consensus.cc:2229] T 241b075a469f4082a2085a7897a33577 P 6399a55532ef4ee382a1a3111bf92a85 [term 1 LEADER]: Raft consensus shutting down.
I20260811 19:02:46.533897 29239 raft_consensus.cc:2258] T 241b075a469f4082a2085a7897a33577 P 6399a55532ef4ee382a1a3111bf92a85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:02:46.536250 28332 catalog_manager.cc:4841] TS 89d1759dc0a94e95a542ac06aa4888de (127.29.112.130:44551): tablet 241b075a469f4082a2085a7897a33577 (table test-table [id=e92593f13b56490c80662157df8bf4a0]) successfully deleted
I20260811 19:02:46.537642 29239 ts_tablet_manager.cc:1905] T 241b075a469f4082a2085a7897a33577 P 6399a55532ef4ee382a1a3111bf92a85: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 11 19:02:46 dist-test-slave-h6n0 krb5kdc[28278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474962, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.129@KRBTEST.COM
I20260811 19:02:46.538751 29234 ts_tablet_manager.cc:1918] T 241b075a469f4082a2085a7897a33577 P 20cfeed44e8041cbbbedae11c0e3ba07: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:02:46.538978 29234 log.cc:1196] T 241b075a469f4082a2085a7897a33577 P 20cfeed44e8041cbbbedae11c0e3ba07: Deleting WAL directory at /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/minicluster-data/ts-2/wal/wals/241b075a469f4082a2085a7897a33577
I20260811 19:02:46.540007 29234 ts_tablet_manager.cc:1939] T 241b075a469f4082a2085a7897a33577 P 20cfeed44e8041cbbbedae11c0e3ba07: Deleting consensus metadata
I20260811 19:02:46.543076 28330 catalog_manager.cc:4841] TS 20cfeed44e8041cbbbedae11c0e3ba07 (127.29.112.131:36637): tablet 241b075a469f4082a2085a7897a33577 (table test-table [id=e92593f13b56490c80662157df8bf4a0]) successfully deleted
I20260811 19:02:46.547547 29193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:46.511586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42790 (local address 127.29.112.129:39721)
0811 19:02:46.511828 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:46.511849 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:46.518787 (+ 6938us) server_negotiation.cc:407] Connection header received
0811 19:02:46.520273 (+ 1486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:46.520280 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:46.520386 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:46.520468 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:02:46.524813 (+ 4345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:46.526228 (+ 1415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:46.531369 (+ 5141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:46.532537 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:46.532734 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:46.539864 (+ 7130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:02:46.539885 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:02:46.539905 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:02:46.539966 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:02:46.543842 (+ 3876us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:46.544657 (+ 815us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:46.544665 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:46.544667 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:46.544839 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:46.545255 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:46.545259 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:46.545261 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:46.545915 (+ 654us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:02:46.546449 (+ 534us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:46.546802 (+ 353us) server_negotiation.cc:299] Negotiation successful
0811 19:02:46.547094 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 19:02:46.548383 28691 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:02:46.685333 29239 ts_tablet_manager.cc:1918] T 241b075a469f4082a2085a7897a33577 P 6399a55532ef4ee382a1a3111bf92a85: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:02:46.685616 29239 log.cc:1196] T 241b075a469f4082a2085a7897a33577 P 6399a55532ef4ee382a1a3111bf92a85: Deleting WAL directory at /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474615205622-30146-0/minicluster-data/ts-0/wal/wals/241b075a469f4082a2085a7897a33577
I20260811 19:02:46.686095 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 28470
I20260811 19:02:46.686290 29239 ts_tablet_manager.cc:1939] T 241b075a469f4082a2085a7897a33577 P 6399a55532ef4ee382a1a3111bf92a85: Deleting consensus metadata
I20260811 19:02:46.719955 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 28709
W20260811 19:02:46.719825 28330 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.112.129:39721 (error 108)
I20260811 19:02:46.721930 29244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:46.721212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:39721 (local address 127.0.0.1:42806)
0811 19:02:46.721654 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:46.721757 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.129:39721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":248,"threads_started":1}
W20260811 19:02:46.722155 28330 catalog_manager.cc:4584] TS 6399a55532ef4ee382a1a3111bf92a85 (127.29.112.129:39721): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 241b075a469f4082a2085a7897a33577: Network error: Client connection negotiation failed: client connection to 127.29.112.129:39721: connect: Connection refused (error 111)
Aug 11 19:02:46 dist-test-slave-h6n0 krb5kdc[28278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474962, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.130@KRBTEST.COM
I20260811 19:02:46.741155 29189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:46.720936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48774 (local address 127.29.112.130:44551)
0811 19:02:46.721140 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:46.721146 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:46.721168 (+ 22us) server_negotiation.cc:407] Connection header received
0811 19:02:46.721347 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:46.721352 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:46.721446 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:46.721561 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:02:46.723170 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:46.723923 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:46.725337 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:46.725735 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:46.725877 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:46.731212 (+ 5335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:02:46.731233 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:02:46.731250 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:02:46.731294 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:02:46.736325 (+ 5031us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:46.737497 (+ 1172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:46.737508 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:46.737511 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:46.737796 (+ 285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:46.738361 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:46.738365 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:46.738368 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:46.739408 (+ 1040us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:02:46.739727 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:46.740108 (+ 381us) server_negotiation.cc:299] Negotiation successful
0811 19:02:46.740511 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 19:02:46.741948 28926 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:02:46.777076 29244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:46.776377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:39721 (local address 127.0.0.1:42810)
0811 19:02:46.776727 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:46.776861 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.129:39721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150}
I20260811 19:02:46.779049 29244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:46.778457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:39721 (local address 127.0.0.1:42814)
0811 19:02:46.778753 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:46.778853 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.129:39721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154}
I20260811 19:02:46.840322 29244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:46.839720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:39721 (local address 127.0.0.1:42816)
0811 19:02:46.840050 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:46.840156 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.129:39721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128}
I20260811 19:02:46.841518 29244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:46.841115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:39721 (local address 127.0.0.1:42826)
0811 19:02:46.841272 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:46.841353 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.129:39721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260811 19:02:46.856487 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 28709
I20260811 19:02:46.889950 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 28943
Aug 11 19:02:46 dist-test-slave-h6n0 krb5kdc[28278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474962, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.131@KRBTEST.COM
I20260811 19:02:46.910579 29246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:46.891605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60358 (local address 127.29.112.131:36637)
0811 19:02:46.892438 (+ 833us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:46.892444 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:46.892464 (+ 20us) server_negotiation.cc:407] Connection header received
0811 19:02:46.892535 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:46.892539 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:46.892608 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:46.892896 (+ 288us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:02:46.894305 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:46.895137 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:46.896577 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:46.896975 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:46.897115 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:46.903413 (+ 6298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:02:46.903436 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:02:46.903455 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:02:46.903508 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:02:46.906991 (+ 3483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:46.907964 (+ 973us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:46.907973 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:46.907975 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:46.908159 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:46.908616 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:46.908621 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:46.908625 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:46.909061 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:02:46.909397 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:46.909782 (+ 385us) server_negotiation.cc:299] Negotiation successful
0811 19:02:46.910056 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":721,"thread_start_us":331,"threads_started":1}
I20260811 19:02:46.911448 29151 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:02:46.922493 29244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:46.921960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:39721 (local address 127.0.0.1:42842)
0811 19:02:46.922206 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:46.922319 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.129:39721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260811 19:02:46.924232 29244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:46.923751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:39721 (local address 127.0.0.1:42854)
0811 19:02:46.923978 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:46.924069 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.129:39721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
I20260811 19:02:47.022624 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 28943
I20260811 19:02:47.059392 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 28298
I20260811 19:02:47.060897 28418 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:02:47.202173 29244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:47.201287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:39721 (local address 127.0.0.1:42866)
0811 19:02:47.201597 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:47.201794 (+ 197us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.129:39721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260811 19:02:47.202571 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 28298
2026-08-11T19:02:47Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (6324 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-11T19:02:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:02:47Z Disabled control of system clock
I20260811 19:02:47.351343 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474615205622-30146-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474615205622-30146-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474615205622-30146-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.190:39981
--webserver_interface=127.29.112.190
--webserver_port=0
--builtin_ntp_servers=127.29.112.148:42765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.190:39981
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 19:02:47.841558 29255 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:02:47.841945 29255 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:02:47.842124 29255 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:02:47.850688 29255 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:02:47.850833 29255 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:02:47.850874 29255 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:02:47.850986 29255 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:02:47.851074 29255 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:02:47.864032 29255 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:42765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474615205622-30146-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474615205622-30146-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.112.190:39981
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.190:39981
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.112.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474615205622-30146-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:02:47.865978 29255 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:02:47.868742 29255 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:02:47.884893 29263 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:02:47.885280 29268 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:02:47.885254 29265 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:02:47.885906 29255 server_base.cc:1034] running on GCE node
I20260811 19:02:47.886876 29255 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:02:47.888749 29255 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:02:47.890007 29255 hybrid_clock.cc:648] HybridClock initialized: now 1786474967889928 us; error 65 us; skew 500 ppm
I20260811 19:02:47.890489 29255 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 19:02:47.893561 29255 webserver.cc:460] Webserver started at http://127.29.112.190:46741/ using document root <none> and password file <none>
I20260811 19:02:47.895403 29255 fs_manager.cc:359] Metadata directory not provided
I20260811 19:02:47.895640 29255 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:02:47.896315 29255 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:02:47.900701 29255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474615205622-30146-0/minicluster-data/master-0/data/instance:
uuid: "7969625228604a84b825a20cb1580f03"
format_stamp: "Formatted at 2026-08-11 19:02:47 on dist-test-slave-h6n0"
server_key: "d0f1a39286cd234bb559405af3b5fa5f"
server_key_iv: "2962fdcf29baddcf0f4fb4e4bd32c3e4"
server_key_version: "encryptionkey@0"
I20260811 19:02:47.901777 29255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474615205622-30146-0/minicluster-data/master-0/wal/instance:
uuid: "7969625228604a84b825a20cb1580f03"
format_stamp: "Formatted at 2026-08-11 19:02:47 on dist-test-slave-h6n0"
server_key: "d0f1a39286cd234bb559405af3b5fa5f"
server_key_iv: "2962fdcf29baddcf0f4fb4e4bd32c3e4"
server_key_version: "encryptionkey@0"
I20260811 19:02:47.908070 29255 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.003s
I20260811 19:02:47.912976 29277 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:47.915586 29255 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260811 19:02:47.915884 29255 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474615205622-30146-0/minicluster-data/master-0/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474615205622-30146-0/minicluster-data/master-0/wal
uuid: "7969625228604a84b825a20cb1580f03"
format_stamp: "Formatted at 2026-08-11 19:02:47 on dist-test-slave-h6n0"
server_key: "d0f1a39286cd234bb559405af3b5fa5f"
server_key_iv: "2962fdcf29baddcf0f4fb4e4bd32c3e4"
server_key_version: "encryptionkey@0"
I20260811 19:02:47.916168 29255 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474615205622-30146-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474615205622-30146-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474615205622-30146-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:02:47.940798 29255 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:02:47.941807 29255 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:02:47.942121 29255 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:02:47.972317 29371 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.190:39981 every 8 connection(s)
I20260811 19:02:47.972466 29255 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.190:39981
I20260811 19:02:47.974654 29255 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
I20260811 19:02:47.980068 29372 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:02:47.983916 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 29255
I20260811 19:02:47.984133 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474615205622-30146-0/minicluster-data/master-0/wal/instance
I20260811 19:02:47.984915 30146 external_mini_cluster.cc:1428] Setting key fadb89b8ace709619f736a70d99fd075
I20260811 19:02:47.998224 29372 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7969625228604a84b825a20cb1580f03: Bootstrap starting.
I20260811 19:02:48.004240 29372 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7969625228604a84b825a20cb1580f03: Neither blocks nor log segments found. Creating new log.
I20260811 19:02:48.005389 29378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:47.987301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53016 (local address 127.29.112.190:39981)
0811 19:02:47.988524 (+ 1223us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:47.988537 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:47.988589 (+ 52us) server_negotiation.cc:407] Connection header received
0811 19:02:47.990473 (+ 1884us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:47.990499 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:47.991520 (+ 1021us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:47.992117 (+ 597us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:02:47.993404 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:47.995149 (+ 1745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:47.998135 (+ 2986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:47.999158 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:47.999322 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:48.001552 (+ 2230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:02:48.001562 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:02:48.001579 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:02:48.002292 (+ 713us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:02:48.002420 (+ 128us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:02:48.002447 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:02:48.002655 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:48.003074 (+ 419us) server_negotiation.cc:299] Negotiation successful
0811 19:02:48.003817 (+ 743us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":770,"thread_start_us":471,"threads_started":1}
I20260811 19:02:48.005925 29372 log.cc:824] T 00000000000000000000000000000000 P 7969625228604a84b825a20cb1580f03: Log is configured to *not* fsync() on all Append() calls
I20260811 19:02:48.011914 29372 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7969625228604a84b825a20cb1580f03: No bootstrap required, opened a new log
I20260811 19:02:48.025880 29372 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7969625228604a84b825a20cb1580f03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7969625228604a84b825a20cb1580f03" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 39981 } }
I20260811 19:02:48.026432 29372 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7969625228604a84b825a20cb1580f03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:02:48.026585 29372 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7969625228604a84b825a20cb1580f03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7969625228604a84b825a20cb1580f03, State: Initialized, Role: FOLLOWER
I20260811 19:02:48.027467 29372 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7969625228604a84b825a20cb1580f03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7969625228604a84b825a20cb1580f03" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 39981 } }
I20260811 19:02:48.027782 29372 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7969625228604a84b825a20cb1580f03 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:02:48.027993 29372 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7969625228604a84b825a20cb1580f03 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:02:48.028204 29372 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7969625228604a84b825a20cb1580f03 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:02:48.031225 29372 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7969625228604a84b825a20cb1580f03 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7969625228604a84b825a20cb1580f03" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 39981 } }
I20260811 19:02:48.031898 29372 leader_election.cc:302] T 00000000000000000000000000000000 P 7969625228604a84b825a20cb1580f03 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7969625228604a84b825a20cb1580f03; no voters:
I20260811 19:02:48.033102 29372 leader_election.cc:288] T 00000000000000000000000000000000 P 7969625228604a84b825a20cb1580f03 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:02:48.033341 29381 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7969625228604a84b825a20cb1580f03 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:02:48.034519 29381 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7969625228604a84b825a20cb1580f03 [term 1 LEADER]: Becoming Leader. State: Replica: 7969625228604a84b825a20cb1580f03, State: Running, Role: LEADER
I20260811 19:02:48.035204 29381 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7969625228604a84b825a20cb1580f03 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7969625228604a84b825a20cb1580f03" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 39981 } }
I20260811 19:02:48.036432 29372 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7969625228604a84b825a20cb1580f03 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:02:48.039134 29383 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7969625228604a84b825a20cb1580f03 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7969625228604a84b825a20cb1580f03" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7969625228604a84b825a20cb1580f03" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 39981 } } }
I20260811 19:02:48.039896 29383 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7969625228604a84b825a20cb1580f03 [sys.catalog]: This master's current role is: LEADER
I20260811 19:02:48.039250 29382 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7969625228604a84b825a20cb1580f03 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7969625228604a84b825a20cb1580f03. Latest consensus state: current_term: 1 leader_uuid: "7969625228604a84b825a20cb1580f03" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7969625228604a84b825a20cb1580f03" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 39981 } } }
I20260811 19:02:48.040931 29382 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7969625228604a84b825a20cb1580f03 [sys.catalog]: This master's current role is: LEADER
I20260811 19:02:48.043332 29389 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:02:48.050323 29389 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:02:48.064309 29389 catalog_manager.cc:1347] Generated new cluster ID: bccada9081474544908e48d0e94043a2
I20260811 19:02:48.064463 29389 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:02:48.093871 29389 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:02:48.096437 29389 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:02:48.109215 29389 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7969625228604a84b825a20cb1580f03: Generated new TSK 0
I20260811 19:02:48.110459 29389 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 19:02:48.118506 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474615205622-30146-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474615205622-30146-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474615205622-30146-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474615205622-30146-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.129:0
--local_ip_for_outbound_sockets=127.29.112.129
--webserver_interface=127.29.112.129
--webserver_port=0
--tserver_master_addrs=127.29.112.190:39981
--builtin_ntp_servers=127.29.112.148:42765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 19:02:48.627585 29414 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:02:48.627961 29414 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:02:48.628096 29414 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:02:48.637418 29414 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:02:48.637624 29414 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:02:48.637806 29414 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.129
I20260811 19:02:48.650313 29414 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:42765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474615205622-30146-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474615205622-30146-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474615205622-30146-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.112.129
--webserver_port=0
--tserver_master_addrs=127.29.112.190:39981
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.129
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474615205622-30146-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:02:48.652348 29414 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:02:48.655647 29414 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:02:48.674739 29427 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:02:48.675340 29433 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:02:48.675345 29425 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:02:48.675727 29414 server_base.cc:1034] running on GCE node
I20260811 19:02:48.676673 29414 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:02:48.677860 29414 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:02:48.681427 29414 hybrid_clock.cc:648] HybridClock initialized: now 1786474968681297 us; error 95 us; skew 500 ppm
I20260811 19:02:48.682142 29414 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 19:02:48.692939 29414 webserver.cc:460] Webserver started at http://127.29.112.129:39909/ using document root <none> and password file <none>
I20260811 19:02:48.693877 29414 fs_manager.cc:359] Metadata directory not provided
I20260811 19:02:48.693996 29414 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:02:48.694377 29414 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:02:48.699463 29414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474615205622-30146-0/minicluster-data/ts-0/data/instance:
uuid: "947733e999774d6fa74535e80bc230c2"
format_stamp: "Formatted at 2026-08-11 19:02:48 on dist-test-slave-h6n0"
server_key: "7446d9e910448370ff9a412aa4ae8b6e"
server_key_iv: "3e6a833db8ceaa4bd4e3d7a57bb40f02"
server_key_version: "encryptionkey@0"
I20260811 19:02:48.700750 29414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474615205622-30146-0/minicluster-data/ts-0/wal/instance:
uuid: "947733e999774d6fa74535e80bc230c2"
format_stamp: "Formatted at 2026-08-11 19:02:48 on dist-test-slave-h6n0"
server_key: "7446d9e910448370ff9a412aa4ae8b6e"
server_key_iv: "3e6a833db8ceaa4bd4e3d7a57bb40f02"
server_key_version: "encryptionkey@0"
I20260811 19:02:48.708078 29414 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.003s sys 0.005s
I20260811 19:02:48.712818 29445 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:48.715870 29414 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.004s
I20260811 19:02:48.716106 29414 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474615205622-30146-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474615205622-30146-0/minicluster-data/ts-0/wal
uuid: "947733e999774d6fa74535e80bc230c2"
format_stamp: "Formatted at 2026-08-11 19:02:48 on dist-test-slave-h6n0"
server_key: "7446d9e910448370ff9a412aa4ae8b6e"
server_key_iv: "3e6a833db8ceaa4bd4e3d7a57bb40f02"
server_key_version: "encryptionkey@0"
I20260811 19:02:48.716426 29414 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474615205622-30146-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474615205622-30146-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474615205622-30146-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:02:48.741276 29414 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:02:48.742515 29414 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:02:48.743525 29414 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:02:48.745292 29414 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:02:48.747941 29414 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:02:48.748335 29414 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:48.748553 29414 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:02:48.748680 29414 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:48.819684 29414 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.129:40217
I20260811 19:02:48.819729 29627 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.129:40217 every 8 connection(s)
I20260811 19:02:48.823019 29414 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474615205622-30146-0/minicluster-data/ts-0/data/info.pb
I20260811 19:02:48.831688 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 29414
I20260811 19:02:48.832475 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474615205622-30146-0/minicluster-data/ts-0/wal/instance
I20260811 19:02:48.834380 30146 external_mini_cluster.cc:1428] Setting key 5e6cf3c33a6ea95ad5b06b008e84a144
I20260811 19:02:48.843436 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474615205622-30146-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474615205622-30146-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474615205622-30146-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474615205622-30146-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.130:0
--local_ip_for_outbound_sockets=127.29.112.130
--webserver_interface=127.29.112.130
--webserver_port=0
--tserver_master_addrs=127.29.112.190:39981
--builtin_ntp_servers=127.29.112.148:42765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 19:02:48.846581 29631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:48.827375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.129:48581 (local address 127.29.112.190:39981)
0811 19:02:48.827895 (+ 520us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:48.827899 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:48.829062 (+ 1163us) server_negotiation.cc:407] Connection header received
0811 19:02:48.833223 (+ 4161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:48.833229 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:48.833336 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:48.833476 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:02:48.834738 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:48.837768 (+ 3030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:48.841532 (+ 3764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:48.842040 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:48.842587 (+ 547us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:48.845355 (+ 2768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:02:48.845370 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:02:48.845375 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:02:48.845404 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:02:48.845463 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:02:48.845479 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:02:48.845613 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:48.845964 (+ 351us) server_negotiation.cc:299] Negotiation successful
0811 19:02:48.846110 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":417,"thread_start_us":290,"threads_started":1}
I20260811 19:02:48.848891 29632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:48.827789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:39981 (local address 127.29.112.129:48581)
0811 19:02:48.828890 (+ 1101us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:48.828928 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 19:02:48.832870 (+ 3942us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:02:48.833754 (+ 884us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:02:48.833762 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:02:48.833870 (+ 108us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:02:48.834492 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:48.834510 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:48.838440 (+ 3930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:48.838447 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:48.841210 (+ 2763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:48.841229 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:48.842771 (+ 1542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:48.842783 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:48.843146 (+ 363us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:48.844632 (+ 1486us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 19:02:48.844635 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:02:48.844702 (+ 67us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 19:02:48.844858 (+ 156us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:02:48.845623 (+ 765us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:02:48.845654 (+ 31us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:02:48.845671 (+ 17us) client_negotiation.cc:769] Sending connection context
0811 19:02:48.845935 (+ 264us) client_negotiation.cc:240] Negotiation successful
0811 19:02:48.846472 (+ 537us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":607,"thread_start_us":374,"threads_started":1}
I20260811 19:02:48.852106 29628 heartbeater.cc:347] Connected to a master server at 127.29.112.190:39981
I20260811 19:02:48.852710 29628 heartbeater.cc:464] Registering TS with master...
I20260811 19:02:48.853791 29628 heartbeater.cc:511] Master 127.29.112.190:39981 requested a full tablet report, sending...
I20260811 19:02:48.856702 29305 ts_manager.cc:194] Registered new tserver with Master: 947733e999774d6fa74535e80bc230c2 (127.29.112.129:40217)
I20260811 19:02:48.860873 29305 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.112.129:48581
W20260811 19:02:49.311885 29635 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:02:49.312218 29635 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:02:49.312400 29635 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:02:49.320044 29635 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:02:49.320173 29635 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:02:49.320350 29635 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.130
I20260811 19:02:49.330875 29635 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:42765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474615205622-30146-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474615205622-30146-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474615205622-30146-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.112.130
--webserver_port=0
--tserver_master_addrs=127.29.112.190:39981
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.130
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474615205622-30146-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:02:49.332460 29635 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:02:49.334650 29635 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:02:49.353413 29641 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:02:49.353412 29642 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:02:49.353514 29648 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:02:49.355473 29635 server_base.cc:1034] running on GCE node
I20260811 19:02:49.356369 29635 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:02:49.358165 29635 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:02:49.359541 29635 hybrid_clock.cc:648] HybridClock initialized: now 1786474969359407 us; error 114 us; skew 500 ppm
I20260811 19:02:49.360131 29635 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 19:02:49.363054 29635 webserver.cc:460] Webserver started at http://127.29.112.130:40689/ using document root <none> and password file <none>
I20260811 19:02:49.363831 29635 fs_manager.cc:359] Metadata directory not provided
I20260811 19:02:49.364231 29635 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:02:49.364650 29635 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:02:49.368046 29635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474615205622-30146-0/minicluster-data/ts-1/data/instance:
uuid: "0416106e08e745a8af7a1718353ee95a"
format_stamp: "Formatted at 2026-08-11 19:02:49 on dist-test-slave-h6n0"
server_key: "68fc3edf38d9c18bc52e6714b00d7dc6"
server_key_iv: "84706a5059b2a9ec956b48b1a3b4240b"
server_key_version: "encryptionkey@0"
I20260811 19:02:49.369184 29635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474615205622-30146-0/minicluster-data/ts-1/wal/instance:
uuid: "0416106e08e745a8af7a1718353ee95a"
format_stamp: "Formatted at 2026-08-11 19:02:49 on dist-test-slave-h6n0"
server_key: "68fc3edf38d9c18bc52e6714b00d7dc6"
server_key_iv: "84706a5059b2a9ec956b48b1a3b4240b"
server_key_version: "encryptionkey@0"
I20260811 19:02:49.375273 29635 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260811 19:02:49.379966 29654 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:49.381929 29635 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260811 19:02:49.382174 29635 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474615205622-30146-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474615205622-30146-0/minicluster-data/ts-1/wal
uuid: "0416106e08e745a8af7a1718353ee95a"
format_stamp: "Formatted at 2026-08-11 19:02:49 on dist-test-slave-h6n0"
server_key: "68fc3edf38d9c18bc52e6714b00d7dc6"
server_key_iv: "84706a5059b2a9ec956b48b1a3b4240b"
server_key_version: "encryptionkey@0"
I20260811 19:02:49.383096 29635 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474615205622-30146-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474615205622-30146-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474615205622-30146-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:02:49.419162 29635 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:02:49.421146 29635 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:02:49.421658 29635 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:02:49.423118 29635 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:02:49.425972 29635 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:02:49.426134 29635 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:49.426301 29635 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:02:49.426409 29635 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:49.479830 29635 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.130:43589
I20260811 19:02:49.479900 29860 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.130:43589 every 8 connection(s)
I20260811 19:02:49.482407 29635 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474615205622-30146-0/minicluster-data/ts-1/data/info.pb
I20260811 19:02:49.487689 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 29635
I20260811 19:02:49.487898 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474615205622-30146-0/minicluster-data/ts-1/wal/instance
I20260811 19:02:49.488686 30146 external_mini_cluster.cc:1428] Setting key 42d614f512f3eba1ef044d3e9a2757ec
I20260811 19:02:49.492959 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474615205622-30146-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474615205622-30146-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474615205622-30146-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474615205622-30146-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.131:0
--local_ip_for_outbound_sockets=127.29.112.131
--webserver_interface=127.29.112.131
--webserver_port=0
--tserver_master_addrs=127.29.112.190:39981
--builtin_ntp_servers=127.29.112.148:42765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 19:02:49.506738 29864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:49.487091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.130:36505 (local address 127.29.112.190:39981)
0811 19:02:49.487987 (+ 896us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:49.487994 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:49.490025 (+ 2031us) server_negotiation.cc:407] Connection header received
0811 19:02:49.492981 (+ 2956us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:49.492992 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:49.493135 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:49.493447 (+ 312us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:02:49.495542 (+ 2095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:49.497521 (+ 1979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:49.500897 (+ 3376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:49.501972 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:49.502380 (+ 408us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:49.504661 (+ 2281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:02:49.504670 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:02:49.504678 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:02:49.504712 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:02:49.504772 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:02:49.504794 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:02:49.505114 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:49.505924 (+ 810us) server_negotiation.cc:299] Negotiation successful
0811 19:02:49.506081 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":749,"thread_start_us":625,"threads_started":1}
I20260811 19:02:49.508093 29865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:49.487712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:39981 (local address 127.29.112.130:36505)
0811 19:02:49.489656 (+ 1944us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:49.489742 (+ 86us) client_negotiation.cc:174] Beginning negotiation
0811 19:02:49.492546 (+ 2804us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:02:49.493704 (+ 1158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:02:49.493713 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:02:49.493826 (+ 113us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:02:49.495193 (+ 1367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:49.495222 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:49.497912 (+ 2690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:49.497918 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:49.500591 (+ 2673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:49.500622 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:49.502493 (+ 1871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:49.502501 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:49.503204 (+ 703us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:49.504140 (+ 936us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 19:02:49.504143 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:02:49.504194 (+ 51us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 19:02:49.504308 (+ 114us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:02:49.505519 (+ 1211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:02:49.505534 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:02:49.505545 (+ 11us) client_negotiation.cc:769] Sending connection context
0811 19:02:49.505833 (+ 288us) client_negotiation.cc:240] Negotiation successful
0811 19:02:49.506363 (+ 530us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":971,"thread_start_us":614,"threads_started":1}
I20260811 19:02:49.512025 29861 heartbeater.cc:347] Connected to a master server at 127.29.112.190:39981
I20260811 19:02:49.512634 29861 heartbeater.cc:464] Registering TS with master...
I20260811 19:02:49.513943 29861 heartbeater.cc:511] Master 127.29.112.190:39981 requested a full tablet report, sending...
I20260811 19:02:49.516747 29305 ts_manager.cc:194] Registered new tserver with Master: 0416106e08e745a8af7a1718353ee95a (127.29.112.130:43589)
I20260811 19:02:49.518100 29305 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.112.130:36505
I20260811 19:02:49.867220 29628 heartbeater.cc:503] Master 127.29.112.190:39981 was elected leader, sending a full tablet report...
W20260811 19:02:49.938808 29866 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:02:49.939617 29866 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:02:49.939806 29866 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:02:49.948375 29866 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:02:49.948868 29866 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:02:49.949247 29866 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.131
I20260811 19:02:49.960136 29866 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:42765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474615205622-30146-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474615205622-30146-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474615205622-30146-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.112.131
--webserver_port=0
--tserver_master_addrs=127.29.112.190:39981
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.131
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474615205622-30146-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:02:49.963269 29866 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:02:49.966459 29866 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:02:49.984673 29874 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:02:49.984793 29872 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:02:49.984771 29879 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:02:49.986050 29866 server_base.cc:1034] running on GCE node
I20260811 19:02:49.986842 29866 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:02:49.988142 29866 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:02:49.989943 29866 hybrid_clock.cc:648] HybridClock initialized: now 1786474969989834 us; error 40 us; skew 500 ppm
I20260811 19:02:49.990765 29866 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 19:02:49.993765 29866 webserver.cc:460] Webserver started at http://127.29.112.131:38411/ using document root <none> and password file <none>
I20260811 19:02:49.995090 29866 fs_manager.cc:359] Metadata directory not provided
I20260811 19:02:49.995427 29866 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:02:49.995766 29866 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:02:49.999007 29866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474615205622-30146-0/minicluster-data/ts-2/data/instance:
uuid: "d712a19f65bf47fc99dba8a9970b44b7"
format_stamp: "Formatted at 2026-08-11 19:02:49 on dist-test-slave-h6n0"
server_key: "028e354b83b695abf2b09c0e2d019c3e"
server_key_iv: "8aa8a69ea3abde6ef054c7903ecc0151"
server_key_version: "encryptionkey@0"
I20260811 19:02:49.999820 29866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474615205622-30146-0/minicluster-data/ts-2/wal/instance:
uuid: "d712a19f65bf47fc99dba8a9970b44b7"
format_stamp: "Formatted at 2026-08-11 19:02:49 on dist-test-slave-h6n0"
server_key: "028e354b83b695abf2b09c0e2d019c3e"
server_key_iv: "8aa8a69ea3abde6ef054c7903ecc0151"
server_key_version: "encryptionkey@0"
I20260811 19:02:50.007225 29866 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.001s
I20260811 19:02:50.013125 29891 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:50.015659 29866 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260811 19:02:50.016065 29866 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474615205622-30146-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474615205622-30146-0/minicluster-data/ts-2/wal
uuid: "d712a19f65bf47fc99dba8a9970b44b7"
format_stamp: "Formatted at 2026-08-11 19:02:49 on dist-test-slave-h6n0"
server_key: "028e354b83b695abf2b09c0e2d019c3e"
server_key_iv: "8aa8a69ea3abde6ef054c7903ecc0151"
server_key_version: "encryptionkey@0"
I20260811 19:02:50.016404 29866 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474615205622-30146-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474615205622-30146-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474615205622-30146-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:02:50.056723 29866 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:02:50.057753 29866 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:02:50.058132 29866 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:02:50.060387 29866 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:02:50.062325 29866 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:02:50.062448 29866 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:50.062628 29866 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:02:50.062747 29866 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:50.115114 29866 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.131:34729
I20260811 19:02:50.115127 30089 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.131:34729 every 8 connection(s)
I20260811 19:02:50.117735 29866 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474615205622-30146-0/minicluster-data/ts-2/data/info.pb
I20260811 19:02:50.125085 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 29866
I20260811 19:02:50.125317 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474615205622-30146-0/minicluster-data/ts-2/wal/instance
I20260811 19:02:50.125890 30146 external_mini_cluster.cc:1428] Setting key 28a41f61a99cbf81d89ab624072bb614
I20260811 19:02:50.142735 30094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:50.123325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.131:49837 (local address 127.29.112.190:39981)
0811 19:02:50.124146 (+ 821us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:50.124153 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:50.125567 (+ 1414us) server_negotiation.cc:407] Connection header received
0811 19:02:50.129131 (+ 3564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:50.129138 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:50.129505 (+ 367us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:50.129638 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:02:50.131924 (+ 2286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:50.133326 (+ 1402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:50.136328 (+ 3002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:50.136976 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:50.137281 (+ 305us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:50.140812 (+ 3531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:02:50.140820 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:02:50.140824 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:02:50.140854 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:02:50.140931 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:02:50.140948 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:02:50.141042 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:50.141781 (+ 739us) server_negotiation.cc:299] Negotiation successful
0811 19:02:50.141957 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":664,"thread_start_us":368,"threads_started":1}
I20260811 19:02:50.144145 30101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:50.124094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:39981 (local address 127.29.112.131:49837)
0811 19:02:50.125316 (+ 1222us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:50.125407 (+ 91us) client_negotiation.cc:174] Beginning negotiation
0811 19:02:50.128482 (+ 3075us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:02:50.130123 (+ 1641us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:02:50.130137 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:02:50.130406 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:02:50.131555 (+ 1149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:50.131581 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:50.133608 (+ 2027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:50.133614 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:50.136044 (+ 2430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:50.136069 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:50.137620 (+ 1551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:50.137632 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:50.138495 (+ 863us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:50.140092 (+ 1597us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 19:02:50.140096 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:02:50.140281 (+ 185us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 19:02:50.140441 (+ 160us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:02:50.141111 (+ 670us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:02:50.141124 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:02:50.141135 (+ 11us) client_negotiation.cc:769] Sending connection context
0811 19:02:50.141584 (+ 449us) client_negotiation.cc:240] Negotiation successful
0811 19:02:50.142098 (+ 514us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":626,"thread_start_us":400,"threads_started":1}
I20260811 19:02:50.145845 30090 heartbeater.cc:347] Connected to a master server at 127.29.112.190:39981
I20260811 19:02:50.146226 30090 heartbeater.cc:464] Registering TS with master...
I20260811 19:02:50.147329 30090 heartbeater.cc:511] Master 127.29.112.190:39981 requested a full tablet report, sending...
I20260811 19:02:50.148999 29305 ts_manager.cc:194] Registered new tserver with Master: d712a19f65bf47fc99dba8a9970b44b7 (127.29.112.131:34729)
I20260811 19:02:50.149958 29305 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.112.131:49837
I20260811 19:02:50.160491 30146 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 19:02:50.175092 30094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:50.166448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58114 (local address 127.29.112.190:39981)
0811 19:02:50.166660 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:50.166665 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:50.167062 (+ 397us) server_negotiation.cc:407] Connection header received
0811 19:02:50.167231 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:50.167236 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:50.167307 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:50.167408 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:02:50.168182 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:50.169035 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:50.171485 (+ 2450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:50.171908 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:50.172019 (+ 111us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:50.173691 (+ 1672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:02:50.173697 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:02:50.173701 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:02:50.173725 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:02:50.173763 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:02:50.173778 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:02:50.173922 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:50.174735 (+ 813us) server_negotiation.cc:299] Negotiation successful
0811 19:02:50.174806 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 19:02:50.181455 29305 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58114:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 19:02:50.190901 29305 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 19:02:50.239291 30121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:50.210101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:40217 (local address 127.0.0.1:39822)
0811 19:02:50.227873 (+ 17772us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:50.227893 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 19:02:50.228186 (+ 293us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:02:50.229164 (+ 978us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:02:50.229169 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:02:50.229193 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:02:50.230758 (+ 1565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:50.230779 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:50.232564 (+ 1785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:50.232571 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:50.235498 (+ 2927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:50.235517 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:50.238056 (+ 2539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:50.238064 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:50.238272 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:50.238332 (+ 60us) client_negotiation.cc:769] Sending connection context
0811 19:02:50.238472 (+ 140us) client_negotiation.cc:240] Negotiation successful
0811 19:02:50.238734 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17637,"mutex_wait_us":975,"thread_start_us":15388,"threads_started":1}
I20260811 19:02:50.240557 30128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:50.211108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39822 (local address 127.29.112.129:40217)
0811 19:02:50.217916 (+ 6808us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:50.217930 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:50.228031 (+ 10101us) server_negotiation.cc:407] Connection header received
0811 19:02:50.228399 (+ 368us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:50.228410 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:50.228725 (+ 315us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:50.228919 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:02:50.230984 (+ 2065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:50.232422 (+ 1438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:50.235796 (+ 3374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:50.237819 (+ 2023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:50.238057 (+ 238us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:50.238391 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:50.238580 (+ 189us) server_negotiation.cc:299] Negotiation successful
0811 19:02:50.238762 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6529,"thread_start_us":6239,"threads_started":1}
I20260811 19:02:50.241122 30124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:50.209784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.131:34729 (local address 127.0.0.1:35620)
0811 19:02:50.227012 (+ 17228us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:50.227037 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 19:02:50.227208 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:02:50.228763 (+ 1555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:02:50.228772 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:02:50.228818 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:02:50.229578 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:50.229597 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:50.232028 (+ 2431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:50.232037 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:50.235718 (+ 3681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:50.235736 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:50.239984 (+ 4248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:50.239995 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:50.240329 (+ 334us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:50.240388 (+ 59us) client_negotiation.cc:769] Sending connection context
0811 19:02:50.240504 (+ 116us) client_negotiation.cc:240] Negotiation successful
0811 19:02:50.240711 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17032,"mutex_wait_us":433,"thread_start_us":16744,"threads_started":1}
I20260811 19:02:50.241869 30127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:50.210503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35620 (local address 127.29.112.131:34729)
0811 19:02:50.227622 (+ 17119us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:50.227638 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:50.227705 (+ 67us) server_negotiation.cc:407] Connection header received
0811 19:02:50.228002 (+ 297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:50.228022 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:50.228420 (+ 398us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:50.228731 (+ 311us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:02:50.229920 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:50.231841 (+ 1921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:50.236588 (+ 4747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:50.238910 (+ 2322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:50.239381 (+ 471us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:50.239696 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:50.240990 (+ 1294us) server_negotiation.cc:299] Negotiation successful
0811 19:02:50.241256 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16851,"thread_start_us":16607,"threads_started":1}
I20260811 19:02:50.242159 30123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:50.209503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41342 (local address 127.29.112.130:43589)
0811 19:02:50.228411 (+ 18908us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:50.228420 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:50.228446 (+ 26us) server_negotiation.cc:407] Connection header received
0811 19:02:50.228533 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:50.228539 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:50.228723 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:50.229344 (+ 621us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:02:50.230382 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:50.231991 (+ 1609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:50.236031 (+ 4040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:50.238228 (+ 2197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:50.240598 (+ 2370us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:50.240893 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:50.240995 (+ 102us) server_negotiation.cc:299] Negotiation successful
0811 19:02:50.241430 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18713,"thread_start_us":18566,"threads_started":1}
I20260811 19:02:50.242551 30125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:50.209403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.130:43589 (local address 127.0.0.1:41342)
0811 19:02:50.224857 (+ 15454us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:50.224935 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0811 19:02:50.225207 (+ 272us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:02:50.229059 (+ 3852us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:02:50.229065 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:02:50.229101 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:02:50.230193 (+ 1092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:50.230209 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:50.232189 (+ 1980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:50.232201 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:50.235718 (+ 3517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:50.235736 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:50.238842 (+ 3106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:50.238852 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:50.239093 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:50.239145 (+ 52us) client_negotiation.cc:769] Sending connection context
0811 19:02:50.239452 (+ 307us) client_negotiation.cc:240] Negotiation successful
0811 19:02:50.239635 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14809,"mutex_wait_us":464,"thread_start_us":18172,"threads_started":1}
I20260811 19:02:50.247284 29752 tablet_service.cc:1467] Processing CreateTablet for tablet 421a354d73674e888f046bfb8ca43186 (DEFAULT_TABLE table=test-table [id=5fcdc31b989c46a98d957cecf4691c7a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:02:50.247502 29523 tablet_service.cc:1467] Processing CreateTablet for tablet 421a354d73674e888f046bfb8ca43186 (DEFAULT_TABLE table=test-table [id=5fcdc31b989c46a98d957cecf4691c7a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:02:50.247488 29982 tablet_service.cc:1467] Processing CreateTablet for tablet 421a354d73674e888f046bfb8ca43186 (DEFAULT_TABLE table=test-table [id=5fcdc31b989c46a98d957cecf4691c7a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:02:50.249059 29752 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 421a354d73674e888f046bfb8ca43186. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:02:50.249083 29982 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 421a354d73674e888f046bfb8ca43186. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:02:50.249089 29523 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 421a354d73674e888f046bfb8ca43186. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:02:50.267789 30130 tablet_bootstrap.cc:492] T 421a354d73674e888f046bfb8ca43186 P 0416106e08e745a8af7a1718353ee95a: Bootstrap starting.
I20260811 19:02:50.271476 30136 tablet_bootstrap.cc:492] T 421a354d73674e888f046bfb8ca43186 P 947733e999774d6fa74535e80bc230c2: Bootstrap starting.
I20260811 19:02:50.272352 30130 tablet_bootstrap.cc:654] T 421a354d73674e888f046bfb8ca43186 P 0416106e08e745a8af7a1718353ee95a: Neither blocks nor log segments found. Creating new log.
I20260811 19:02:50.272441 30137 tablet_bootstrap.cc:492] T 421a354d73674e888f046bfb8ca43186 P d712a19f65bf47fc99dba8a9970b44b7: Bootstrap starting.
I20260811 19:02:50.276600 30130 log.cc:824] T 421a354d73674e888f046bfb8ca43186 P 0416106e08e745a8af7a1718353ee95a: Log is configured to *not* fsync() on all Append() calls
I20260811 19:02:50.277972 30137 tablet_bootstrap.cc:654] T 421a354d73674e888f046bfb8ca43186 P d712a19f65bf47fc99dba8a9970b44b7: Neither blocks nor log segments found. Creating new log.
I20260811 19:02:50.278896 30136 tablet_bootstrap.cc:654] T 421a354d73674e888f046bfb8ca43186 P 947733e999774d6fa74535e80bc230c2: Neither blocks nor log segments found. Creating new log.
I20260811 19:02:50.279503 30137 log.cc:824] T 421a354d73674e888f046bfb8ca43186 P d712a19f65bf47fc99dba8a9970b44b7: Log is configured to *not* fsync() on all Append() calls
I20260811 19:02:50.280671 30136 log.cc:824] T 421a354d73674e888f046bfb8ca43186 P 947733e999774d6fa74535e80bc230c2: Log is configured to *not* fsync() on all Append() calls
I20260811 19:02:50.286254 30137 tablet_bootstrap.cc:492] T 421a354d73674e888f046bfb8ca43186 P d712a19f65bf47fc99dba8a9970b44b7: No bootstrap required, opened a new log
I20260811 19:02:50.286626 30130 tablet_bootstrap.cc:492] T 421a354d73674e888f046bfb8ca43186 P 0416106e08e745a8af7a1718353ee95a: No bootstrap required, opened a new log
I20260811 19:02:50.286661 30137 ts_tablet_manager.cc:1397] T 421a354d73674e888f046bfb8ca43186 P d712a19f65bf47fc99dba8a9970b44b7: Time spent bootstrapping tablet: real 0.014s user 0.006s sys 0.007s
I20260811 19:02:50.286691 30136 tablet_bootstrap.cc:492] T 421a354d73674e888f046bfb8ca43186 P 947733e999774d6fa74535e80bc230c2: No bootstrap required, opened a new log
I20260811 19:02:50.286933 30130 ts_tablet_manager.cc:1397] T 421a354d73674e888f046bfb8ca43186 P 0416106e08e745a8af7a1718353ee95a: Time spent bootstrapping tablet: real 0.019s user 0.008s sys 0.008s
I20260811 19:02:50.286958 30136 ts_tablet_manager.cc:1397] T 421a354d73674e888f046bfb8ca43186 P 947733e999774d6fa74535e80bc230c2: Time spent bootstrapping tablet: real 0.016s user 0.003s sys 0.008s
I20260811 19:02:50.296020 30137 raft_consensus.cc:348] T 421a354d73674e888f046bfb8ca43186 P d712a19f65bf47fc99dba8a9970b44b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0416106e08e745a8af7a1718353ee95a" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 43589 } } peers { permanent_uuid: "d712a19f65bf47fc99dba8a9970b44b7" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 34729 } } peers { permanent_uuid: "947733e999774d6fa74535e80bc230c2" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 40217 } }
I20260811 19:02:50.296639 30137 raft_consensus.cc:374] T 421a354d73674e888f046bfb8ca43186 P d712a19f65bf47fc99dba8a9970b44b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:02:50.296809 30137 raft_consensus.cc:729] T 421a354d73674e888f046bfb8ca43186 P d712a19f65bf47fc99dba8a9970b44b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d712a19f65bf47fc99dba8a9970b44b7, State: Initialized, Role: FOLLOWER
I20260811 19:02:50.297690 30137 consensus_queue.cc:260] T 421a354d73674e888f046bfb8ca43186 P d712a19f65bf47fc99dba8a9970b44b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0416106e08e745a8af7a1718353ee95a" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 43589 } } peers { permanent_uuid: "d712a19f65bf47fc99dba8a9970b44b7" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 34729 } } peers { permanent_uuid: "947733e999774d6fa74535e80bc230c2" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 40217 } }
I20260811 19:02:50.299232 30090 heartbeater.cc:503] Master 127.29.112.190:39981 was elected leader, sending a full tablet report...
I20260811 19:02:50.300122 30137 ts_tablet_manager.cc:1428] T 421a354d73674e888f046bfb8ca43186 P d712a19f65bf47fc99dba8a9970b44b7: Time spent starting tablet: real 0.013s user 0.011s sys 0.001s
I20260811 19:02:50.303678 30130 raft_consensus.cc:348] T 421a354d73674e888f046bfb8ca43186 P 0416106e08e745a8af7a1718353ee95a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0416106e08e745a8af7a1718353ee95a" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 43589 } } peers { permanent_uuid: "d712a19f65bf47fc99dba8a9970b44b7" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 34729 } } peers { permanent_uuid: "947733e999774d6fa74535e80bc230c2" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 40217 } }
I20260811 19:02:50.303681 30136 raft_consensus.cc:348] T 421a354d73674e888f046bfb8ca43186 P 947733e999774d6fa74535e80bc230c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0416106e08e745a8af7a1718353ee95a" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 43589 } } peers { permanent_uuid: "d712a19f65bf47fc99dba8a9970b44b7" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 34729 } } peers { permanent_uuid: "947733e999774d6fa74535e80bc230c2" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 40217 } }
I20260811 19:02:50.304670 30130 raft_consensus.cc:374] T 421a354d73674e888f046bfb8ca43186 P 0416106e08e745a8af7a1718353ee95a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:02:50.304838 30136 raft_consensus.cc:374] T 421a354d73674e888f046bfb8ca43186 P 947733e999774d6fa74535e80bc230c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:02:50.304922 30130 raft_consensus.cc:729] T 421a354d73674e888f046bfb8ca43186 P 0416106e08e745a8af7a1718353ee95a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0416106e08e745a8af7a1718353ee95a, State: Initialized, Role: FOLLOWER
I20260811 19:02:50.305137 30136 raft_consensus.cc:729] T 421a354d73674e888f046bfb8ca43186 P 947733e999774d6fa74535e80bc230c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 947733e999774d6fa74535e80bc230c2, State: Initialized, Role: FOLLOWER
I20260811 19:02:50.306018 30136 consensus_queue.cc:260] T 421a354d73674e888f046bfb8ca43186 P 947733e999774d6fa74535e80bc230c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0416106e08e745a8af7a1718353ee95a" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 43589 } } peers { permanent_uuid: "d712a19f65bf47fc99dba8a9970b44b7" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 34729 } } peers { permanent_uuid: "947733e999774d6fa74535e80bc230c2" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 40217 } }
I20260811 19:02:50.306018 30130 consensus_queue.cc:260] T 421a354d73674e888f046bfb8ca43186 P 0416106e08e745a8af7a1718353ee95a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0416106e08e745a8af7a1718353ee95a" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 43589 } } peers { permanent_uuid: "d712a19f65bf47fc99dba8a9970b44b7" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 34729 } } peers { permanent_uuid: "947733e999774d6fa74535e80bc230c2" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 40217 } }
I20260811 19:02:50.307826 29861 heartbeater.cc:503] Master 127.29.112.190:39981 was elected leader, sending a full tablet report...
I20260811 19:02:50.308984 30136 ts_tablet_manager.cc:1428] T 421a354d73674e888f046bfb8ca43186 P 947733e999774d6fa74535e80bc230c2: Time spent starting tablet: real 0.022s user 0.012s sys 0.008s
I20260811 19:02:50.309818 30130 ts_tablet_manager.cc:1428] T 421a354d73674e888f046bfb8ca43186 P 0416106e08e745a8af7a1718353ee95a: Time spent starting tablet: real 0.023s user 0.015s sys 0.004s
W20260811 19:02:50.327636 29629 tablet.cc:2369] T 421a354d73674e888f046bfb8ca43186 P 947733e999774d6fa74535e80bc230c2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 19:02:50.370716 30093 tablet.cc:2369] T 421a354d73674e888f046bfb8ca43186 P d712a19f65bf47fc99dba8a9970b44b7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 19:02:50.486034 29863 tablet.cc:2369] T 421a354d73674e888f046bfb8ca43186 P 0416106e08e745a8af7a1718353ee95a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 19:02:50.565883 30156 raft_consensus.cc:482] T 421a354d73674e888f046bfb8ca43186 P 0416106e08e745a8af7a1718353ee95a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:02:50.566217 30156 raft_consensus.cc:504] T 421a354d73674e888f046bfb8ca43186 P 0416106e08e745a8af7a1718353ee95a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0416106e08e745a8af7a1718353ee95a" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 43589 } } peers { permanent_uuid: "d712a19f65bf47fc99dba8a9970b44b7" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 34729 } } peers { permanent_uuid: "947733e999774d6fa74535e80bc230c2" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 40217 } }
I20260811 19:02:50.568114 30156 leader_election.cc:288] T 421a354d73674e888f046bfb8ca43186 P 0416106e08e745a8af7a1718353ee95a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d712a19f65bf47fc99dba8a9970b44b7 (127.29.112.131:34729), 947733e999774d6fa74535e80bc230c2 (127.29.112.129:40217)
I20260811 19:02:50.578104 30127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:50.568672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.130:43847 (local address 127.29.112.131:34729)
0811 19:02:50.568897 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:50.568904 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:50.569740 (+ 836us) server_negotiation.cc:407] Connection header received
0811 19:02:50.569995 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:50.570000 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:50.570077 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:50.570183 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:02:50.570816 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:50.571952 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:50.575604 (+ 3652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:50.576668 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:50.577228 (+ 560us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:50.577313 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:50.577639 (+ 326us) server_negotiation.cc:299] Negotiation successful
0811 19:02:50.577776 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 19:02:50.578320 30159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:50.568739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.131:34729 (local address 127.29.112.130:43847)
0811 19:02:50.569600 (+ 861us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:50.569622 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 19:02:50.569817 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:02:50.570201 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:02:50.570205 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:02:50.570231 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:02:50.570659 (+ 428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:50.570669 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:50.572279 (+ 1610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:50.572286 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:50.575277 (+ 2991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:50.575296 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:50.577233 (+ 1937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:50.577245 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:50.577440 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:50.577468 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 19:02:50.577613 (+ 145us) client_negotiation.cc:240] Negotiation successful
0811 19:02:50.577776 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":743,"mutex_wait_us":558,"spinlock_wait_cycles":25856,"thread_start_us":474,"threads_started":1}
I20260811 19:02:50.579547 30128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:50.568743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.130:43567 (local address 127.29.112.129:40217)
0811 19:02:50.568973 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:50.568978 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:50.570275 (+ 1297us) server_negotiation.cc:407] Connection header received
0811 19:02:50.570569 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:50.570578 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:50.570671 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:50.570784 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:02:50.571405 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:50.572220 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:50.575726 (+ 3506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:50.576853 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:50.577133 (+ 280us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:50.577602 (+ 469us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:50.578825 (+ 1223us) server_negotiation.cc:299] Negotiation successful
0811 19:02:50.579195 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 19:02:50.580619 30160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:50.569264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:40217 (local address 127.29.112.130:43567)
0811 19:02:50.570143 (+ 879us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:50.570160 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 19:02:50.570335 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:02:50.570802 (+ 467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:02:50.570807 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:02:50.570832 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:02:50.571216 (+ 384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:50.571226 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:50.572385 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:50.572391 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:50.575320 (+ 2929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:50.575351 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:50.577609 (+ 2258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:50.577622 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:50.578164 (+ 542us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:50.578232 (+ 68us) client_negotiation.cc:769] Sending connection context
0811 19:02:50.579618 (+ 1386us) client_negotiation.cc:240] Negotiation successful
0811 19:02:50.580059 (+ 441us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":761,"mutex_wait_us":250,"thread_start_us":513,"threads_started":1}
I20260811 19:02:50.580873 30014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "421a354d73674e888f046bfb8ca43186" candidate_uuid: "0416106e08e745a8af7a1718353ee95a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d712a19f65bf47fc99dba8a9970b44b7" is_pre_election: true
I20260811 19:02:50.583067 30014 raft_consensus.cc:2454] T 421a354d73674e888f046bfb8ca43186 P d712a19f65bf47fc99dba8a9970b44b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0416106e08e745a8af7a1718353ee95a in term 0.
I20260811 19:02:50.582962 29561 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "421a354d73674e888f046bfb8ca43186" candidate_uuid: "0416106e08e745a8af7a1718353ee95a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "947733e999774d6fa74535e80bc230c2" is_pre_election: true
I20260811 19:02:50.583734 29561 raft_consensus.cc:2454] T 421a354d73674e888f046bfb8ca43186 P 947733e999774d6fa74535e80bc230c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0416106e08e745a8af7a1718353ee95a in term 0.
I20260811 19:02:50.583976 29660 leader_election.cc:302] T 421a354d73674e888f046bfb8ca43186 P 0416106e08e745a8af7a1718353ee95a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0416106e08e745a8af7a1718353ee95a, d712a19f65bf47fc99dba8a9970b44b7; no voters:
I20260811 19:02:50.584599 30156 raft_consensus.cc:2781] T 421a354d73674e888f046bfb8ca43186 P 0416106e08e745a8af7a1718353ee95a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 19:02:50.584792 30156 raft_consensus.cc:482] T 421a354d73674e888f046bfb8ca43186 P 0416106e08e745a8af7a1718353ee95a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 19:02:50.584939 30156 raft_consensus.cc:3037] T 421a354d73674e888f046bfb8ca43186 P 0416106e08e745a8af7a1718353ee95a [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:02:50.587575 30156 raft_consensus.cc:504] T 421a354d73674e888f046bfb8ca43186 P 0416106e08e745a8af7a1718353ee95a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0416106e08e745a8af7a1718353ee95a" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 43589 } } peers { permanent_uuid: "d712a19f65bf47fc99dba8a9970b44b7" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 34729 } } peers { permanent_uuid: "947733e999774d6fa74535e80bc230c2" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 40217 } }
I20260811 19:02:50.588581 30156 leader_election.cc:288] T 421a354d73674e888f046bfb8ca43186 P 0416106e08e745a8af7a1718353ee95a [CANDIDATE]: Term 1 election: Requested vote from peers d712a19f65bf47fc99dba8a9970b44b7 (127.29.112.131:34729), 947733e999774d6fa74535e80bc230c2 (127.29.112.129:40217)
I20260811 19:02:50.589409 30014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "421a354d73674e888f046bfb8ca43186" candidate_uuid: "0416106e08e745a8af7a1718353ee95a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d712a19f65bf47fc99dba8a9970b44b7"
I20260811 19:02:50.589408 29561 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "421a354d73674e888f046bfb8ca43186" candidate_uuid: "0416106e08e745a8af7a1718353ee95a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "947733e999774d6fa74535e80bc230c2"
I20260811 19:02:50.589691 29561 raft_consensus.cc:3037] T 421a354d73674e888f046bfb8ca43186 P 947733e999774d6fa74535e80bc230c2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:02:50.589694 30014 raft_consensus.cc:3037] T 421a354d73674e888f046bfb8ca43186 P d712a19f65bf47fc99dba8a9970b44b7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:02:50.593693 29561 raft_consensus.cc:2454] T 421a354d73674e888f046bfb8ca43186 P 947733e999774d6fa74535e80bc230c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0416106e08e745a8af7a1718353ee95a in term 1.
I20260811 19:02:50.593693 30014 raft_consensus.cc:2454] T 421a354d73674e888f046bfb8ca43186 P d712a19f65bf47fc99dba8a9970b44b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0416106e08e745a8af7a1718353ee95a in term 1.
I20260811 19:02:50.594986 29655 leader_election.cc:302] T 421a354d73674e888f046bfb8ca43186 P 0416106e08e745a8af7a1718353ee95a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0416106e08e745a8af7a1718353ee95a, 947733e999774d6fa74535e80bc230c2; no voters:
I20260811 19:02:50.595618 30156 raft_consensus.cc:2781] T 421a354d73674e888f046bfb8ca43186 P 0416106e08e745a8af7a1718353ee95a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:02:50.596468 30156 raft_consensus.cc:686] T 421a354d73674e888f046bfb8ca43186 P 0416106e08e745a8af7a1718353ee95a [term 1 LEADER]: Becoming Leader. State: Replica: 0416106e08e745a8af7a1718353ee95a, State: Running, Role: LEADER
I20260811 19:02:50.597548 30156 consensus_queue.cc:237] T 421a354d73674e888f046bfb8ca43186 P 0416106e08e745a8af7a1718353ee95a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0416106e08e745a8af7a1718353ee95a" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 43589 } } peers { permanent_uuid: "d712a19f65bf47fc99dba8a9970b44b7" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 34729 } } peers { permanent_uuid: "947733e999774d6fa74535e80bc230c2" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 40217 } }
I20260811 19:02:50.605525 29305 catalog_manager.cc:5495] T 421a354d73674e888f046bfb8ca43186 P 0416106e08e745a8af7a1718353ee95a reported cstate change: term changed from 0 to 1, leader changed from <none> to 0416106e08e745a8af7a1718353ee95a (127.29.112.130). New cstate: current_term: 1 leader_uuid: "0416106e08e745a8af7a1718353ee95a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0416106e08e745a8af7a1718353ee95a" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 43589 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d712a19f65bf47fc99dba8a9970b44b7" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 34729 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "947733e999774d6fa74535e80bc230c2" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 40217 } health_report { overall_health: UNKNOWN } } }
I20260811 19:02:50.709057 30123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:50.699770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41352 (local address 127.29.112.130:43589)
0811 19:02:50.700129 (+ 359us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:50.700135 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:50.701354 (+ 1219us) server_negotiation.cc:407] Connection header received
0811 19:02:50.701718 (+ 364us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:50.701725 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:50.702054 (+ 329us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:50.702460 (+ 406us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:02:50.703473 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:50.704593 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:50.706369 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:50.707020 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:50.707146 (+ 126us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:50.707773 (+ 627us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:02:50.708351 (+ 578us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:02:50.708516 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:50.708728 (+ 212us) server_negotiation.cc:299] Negotiation successful
0811 19:02:50.708776 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 19:02:50.720453 29561 raft_consensus.cc:1261] T 421a354d73674e888f046bfb8ca43186 P 947733e999774d6fa74535e80bc230c2 [term 1 FOLLOWER]: Refusing update from remote peer 0416106e08e745a8af7a1718353ee95a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:02:50.720453 30014 raft_consensus.cc:1261] T 421a354d73674e888f046bfb8ca43186 P d712a19f65bf47fc99dba8a9970b44b7 [term 1 FOLLOWER]: Refusing update from remote peer 0416106e08e745a8af7a1718353ee95a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:02:50.721799 30161 consensus_queue.cc:1035] T 421a354d73674e888f046bfb8ca43186 P 0416106e08e745a8af7a1718353ee95a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d712a19f65bf47fc99dba8a9970b44b7" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 34729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:02:50.722672 30156 consensus_queue.cc:1035] T 421a354d73674e888f046bfb8ca43186 P 0416106e08e745a8af7a1718353ee95a [LEADER]: Connected to new peer: Peer: permanent_uuid: "947733e999774d6fa74535e80bc230c2" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 40217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:02:50.752640 29305 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:58114:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:02:50.753247 29305 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:58114:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:02:50.761247 30173 mvcc.cc:205] Tried to move back new op lower bound from 7317401480055988224 to 7317401479573688320. Current Snapshot: MvccSnapshot[applied={T|T < 7317401480055988224}]
I20260811 19:02:50.761579 29305 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7969625228604a84b825a20cb1580f03: Sending DeleteTablet for 3 replicas of tablet 421a354d73674e888f046bfb8ca43186
I20260811 19:02:50.764066 29752 tablet_service.cc:1514] Processing DeleteTablet for tablet 421a354d73674e888f046bfb8ca43186 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:02:50 UTC) from {username='slave'} at 127.0.0.1:41342
I20260811 19:02:50.764174 29523 tablet_service.cc:1514] Processing DeleteTablet for tablet 421a354d73674e888f046bfb8ca43186 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:02:50 UTC) from {username='slave'} at 127.0.0.1:39822
I20260811 19:02:50.764894 29982 tablet_service.cc:1514] Processing DeleteTablet for tablet 421a354d73674e888f046bfb8ca43186 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:02:50 UTC) from {username='slave'} at 127.0.0.1:35620
I20260811 19:02:50.765565 29522 tablet_service.cc:1514] Processing DeleteTablet for tablet 421a354d73674e888f046bfb8ca43186 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:02:50 UTC) from {username='slave'} at 127.0.0.1:39822
I20260811 19:02:50.766798 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 29414
I20260811 19:02:50.775352 30189 tablet_replica.cc:330] T 421a354d73674e888f046bfb8ca43186 P d712a19f65bf47fc99dba8a9970b44b7: stopping tablet replica
I20260811 19:02:50.776193 30187 tablet_replica.cc:330] T 421a354d73674e888f046bfb8ca43186 P 947733e999774d6fa74535e80bc230c2: stopping tablet replica
I20260811 19:02:50.777288 30187 raft_consensus.cc:2229] T 421a354d73674e888f046bfb8ca43186 P 947733e999774d6fa74535e80bc230c2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:02:50.777748 30189 raft_consensus.cc:2229] T 421a354d73674e888f046bfb8ca43186 P d712a19f65bf47fc99dba8a9970b44b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:02:50.778618 30189 raft_consensus.cc:2258] T 421a354d73674e888f046bfb8ca43186 P d712a19f65bf47fc99dba8a9970b44b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:02:50.779304 30187 raft_consensus.cc:2258] T 421a354d73674e888f046bfb8ca43186 P 947733e999774d6fa74535e80bc230c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:02:50.780939 30128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:50.767709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39830 (local address 127.29.112.129:40217)
0811 19:02:50.767939 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:50.767946 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:50.769704 (+ 1758us) server_negotiation.cc:407] Connection header received
0811 19:02:50.770096 (+ 392us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:50.770105 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:50.770251 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:50.770415 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:02:50.772104 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:50.773652 (+ 1548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:50.776220 (+ 2568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:50.776963 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:50.777080 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:50.778833 (+ 1753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:02:50.778845 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:02:50.778864 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:02:50.779704 (+ 840us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:02:50.779768 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:02:50.779796 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:02:50.779900 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:50.780405 (+ 505us) server_negotiation.cc:299] Negotiation successful
0811 19:02:50.780536 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 19:02:50.782233 29623 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:02:50.783176 30189 ts_tablet_manager.cc:1905] T 421a354d73674e888f046bfb8ca43186 P d712a19f65bf47fc99dba8a9970b44b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:02:50.787103 30188 tablet_replica.cc:330] T 421a354d73674e888f046bfb8ca43186 P 0416106e08e745a8af7a1718353ee95a: stopping tablet replica
I20260811 19:02:50.787703 30188 raft_consensus.cc:2229] T 421a354d73674e888f046bfb8ca43186 P 0416106e08e745a8af7a1718353ee95a [term 1 LEADER]: Raft consensus shutting down.
I20260811 19:02:50.789836 30188 raft_consensus.cc:2258] T 421a354d73674e888f046bfb8ca43186 P 0416106e08e745a8af7a1718353ee95a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:02:50.792232 30189 ts_tablet_manager.cc:1918] T 421a354d73674e888f046bfb8ca43186 P d712a19f65bf47fc99dba8a9970b44b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:02:50.792827 30189 log.cc:1196] T 421a354d73674e888f046bfb8ca43186 P d712a19f65bf47fc99dba8a9970b44b7: Deleting WAL directory at /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474615205622-30146-0/minicluster-data/ts-2/wal/wals/421a354d73674e888f046bfb8ca43186
I20260811 19:02:50.792838 30188 ts_tablet_manager.cc:1905] T 421a354d73674e888f046bfb8ca43186 P 0416106e08e745a8af7a1718353ee95a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:02:50.798040 30189 ts_tablet_manager.cc:1939] T 421a354d73674e888f046bfb8ca43186 P d712a19f65bf47fc99dba8a9970b44b7: Deleting consensus metadata
I20260811 19:02:50.802385 29284 catalog_manager.cc:4841] TS d712a19f65bf47fc99dba8a9970b44b7 (127.29.112.131:34729): tablet 421a354d73674e888f046bfb8ca43186 (table test-table [id=5fcdc31b989c46a98d957cecf4691c7a]) successfully deleted
I20260811 19:02:50.816094 30188 ts_tablet_manager.cc:1918] T 421a354d73674e888f046bfb8ca43186 P 0416106e08e745a8af7a1718353ee95a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:02:50.816428 30188 log.cc:1196] T 421a354d73674e888f046bfb8ca43186 P 0416106e08e745a8af7a1718353ee95a: Deleting WAL directory at /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474615205622-30146-0/minicluster-data/ts-1/wal/wals/421a354d73674e888f046bfb8ca43186
I20260811 19:02:50.817409 30188 ts_tablet_manager.cc:1939] T 421a354d73674e888f046bfb8ca43186 P 0416106e08e745a8af7a1718353ee95a: Deleting consensus metadata
I20260811 19:02:50.820406 29282 catalog_manager.cc:4841] TS 0416106e08e745a8af7a1718353ee95a (127.29.112.130:43589): tablet 421a354d73674e888f046bfb8ca43186 (table test-table [id=5fcdc31b989c46a98d957cecf4691c7a]) successfully deleted
I20260811 19:02:50.908595 30187 ts_tablet_manager.cc:1905] T 421a354d73674e888f046bfb8ca43186 P 947733e999774d6fa74535e80bc230c2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:02:50.914762 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 29414
W20260811 19:02:50.955060 29280 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.112.129:40217 (error 108)
I20260811 19:02:50.955387 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 29635
I20260811 19:02:50.958878 30194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:50.957405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:40217 (local address 127.0.0.1:39846)
0811 19:02:50.958429 (+ 1024us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:50.958668 (+ 239us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.129:40217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":743,"thread_start_us":560,"threads_started":1}
W20260811 19:02:50.959335 29280 catalog_manager.cc:4584] TS 947733e999774d6fa74535e80bc230c2 (127.29.112.129:40217): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 421a354d73674e888f046bfb8ca43186: Network error: Client connection negotiation failed: client connection to 127.29.112.129:40217: connect: Connection refused (error 111)
I20260811 19:02:50.966286 30123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:50.956644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41360 (local address 127.29.112.130:43589)
0811 19:02:50.956971 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:50.956979 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:50.957010 (+ 31us) server_negotiation.cc:407] Connection header received
0811 19:02:50.957144 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:50.957152 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:50.957354 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:50.957530 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:02:50.958592 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:50.960115 (+ 1523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:50.961929 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:50.962888 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:50.963055 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:50.964240 (+ 1185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:02:50.964252 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:02:50.964272 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:02:50.965270 (+ 998us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:02:50.965338 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:02:50.965366 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:02:50.965502 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:50.965704 (+ 202us) server_negotiation.cc:299] Negotiation successful
0811 19:02:50.965838 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164}
I20260811 19:02:50.967201 29854 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:02:51.004284 30194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:51.003715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:40217 (local address 127.0.0.1:39848)
0811 19:02:51.004011 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:51.004131 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.129:40217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147}
I20260811 19:02:51.005609 30194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:51.005174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:40217 (local address 127.0.0.1:39864)
0811 19:02:51.005385 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:51.005461 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.129:40217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260811 19:02:51.013494 30194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:51.012761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:40217 (local address 127.0.0.1:39878)
0811 19:02:51.012979 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:51.013073 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.129:40217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260811 19:02:51.015158 30194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:51.014716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:40217 (local address 127.0.0.1:39892)
0811 19:02:51.014884 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:51.014977 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.129:40217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260811 19:02:51.054519 30194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:51.053848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:40217 (local address 127.0.0.1:39902)
0811 19:02:51.054189 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:51.054312 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.129:40217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158}
I20260811 19:02:51.056388 30194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:51.055517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:40217 (local address 127.0.0.1:39910)
0811 19:02:51.055940 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:51.056102 (+ 162us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.129:40217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219}
I20260811 19:02:51.092698 30194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:51.091935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:40217 (local address 127.0.0.1:39918)
0811 19:02:51.092347 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:51.092478 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.129:40217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181}
I20260811 19:02:51.094058 30194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:51.093584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:40217 (local address 127.0.0.1:39932)
0811 19:02:51.093764 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:51.093858 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.129:40217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260811 19:02:51.111620 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 29635
I20260811 19:02:51.151472 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 29866
I20260811 19:02:51.157444 30194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:51.156883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:40217 (local address 127.0.0.1:39936)
0811 19:02:51.157156 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:51.157268 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.129:40217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
I20260811 19:02:51.159233 30194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:51.158433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:40217 (local address 127.0.0.1:39946)
0811 19:02:51.158847 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:51.158973 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.129:40217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":291}
I20260811 19:02:51.162708 30197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:51.152674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35626 (local address 127.29.112.131:34729)
0811 19:02:51.153742 (+ 1068us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:51.153748 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:51.153765 (+ 17us) server_negotiation.cc:407] Connection header received
0811 19:02:51.153826 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:51.153830 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:51.153923 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:51.154080 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:02:51.155014 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:51.155783 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:51.157284 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:51.158185 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:51.158422 (+ 237us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:51.160512 (+ 2090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:02:51.160522 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:02:51.160539 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:02:51.161193 (+ 654us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:02:51.161250 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:02:51.161274 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:02:51.161417 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:51.161893 (+ 476us) server_negotiation.cc:299] Negotiation successful
0811 19:02:51.162083 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":952,"thread_start_us":821,"threads_started":1}
I20260811 19:02:51.163605 30085 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:02:51.196674 30194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:51.195644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:40217 (local address 127.0.0.1:39958)
0811 19:02:51.196029 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:51.196387 (+ 358us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.129:40217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154}
I20260811 19:02:51.198614 30194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:51.197743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:40217 (local address 127.0.0.1:39964)
0811 19:02:51.198002 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:51.198124 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.129:40217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260811 19:02:51.278880 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 29866
I20260811 19:02:51.319954 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 29255
I20260811 19:02:51.321648 29363 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:02:51.457701 30194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:51.456210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:40217 (local address 127.0.0.1:39970)
0811 19:02:51.457182 (+ 972us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:51.457326 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.129:40217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155}
I20260811 19:02:51.458074 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 29255
2026-08-11T19:02:51Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (4254 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:02:51 dist-test-slave-h6n0 krb5kdc[30207](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:02:51 dist-test-slave-h6n0 krb5kdc[30207](info): set up 2 sockets
Aug 11 19:02:51 dist-test-slave-h6n0 krb5kdc[30207](info): commencing operation
krb5kdc: starting...
W20260811 19:02:53.088205 30146 mini_kdc.cc:121] Time spent starting KDC: real 1.558s user 0.001s sys 0.059s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/krb5kdc/test-user.keytab.
Aug 11 19:02:53 dist-test-slave-h6n0 krb5kdc[30207](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474973, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:02:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:02:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.112.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab.
Entry for principal kudu/127.29.112.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab.
I20260811 19:02:53.544610 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.190:36845
--webserver_interface=127.29.112.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab
--principal=kudu/127.29.112.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:39161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.190:36845
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:02:54.065600 30229 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:02:54.065966 30229 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:02:54.066071 30229 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:02:54.077071 30229 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:02:54.077245 30229 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:02:54.077303 30229 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:02:54.077490 30229 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:02:54.077586 30229 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:02:54.091331 30229 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:39161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.112.190:36845
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab
--principal=kudu/127.29.112.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.190:36845
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.112.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:02:54.093703 30229 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:02:54.095590 30229 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:02:54.110605 30237 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:02:54.110723 30239 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:02:54.110744 30241 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:02:54.111507 30229 server_base.cc:1034] running on GCE node
I20260811 19:02:54.112449 30229 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:02:54.114148 30229 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:02:54.115720 30229 hybrid_clock.cc:648] HybridClock initialized: now 1786474974115643 us; error 134 us; skew 500 ppm
I20260811 19:02:54.116497 30229 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:02:54 dist-test-slave-h6n0 krb5kdc[30207](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474974, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:02:54.126453 30229 init.cc:377] Logged in from keytab as kudu/127.29.112.190@KRBTEST.COM (short username kudu)
I20260811 19:02:54.128717 30229 webserver.cc:460] Webserver started at http://127.29.112.190:42973/ using document root <none> and password file <none>
I20260811 19:02:54.129659 30229 fs_manager.cc:359] Metadata directory not provided
I20260811 19:02:54.129841 30229 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:02:54.130262 30229 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:02:54.134112 30229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/minicluster-data/master-0/data/instance:
uuid: "a0c12cf38d684de287a83812a1bd4d32"
format_stamp: "Formatted at 2026-08-11 19:02:54 on dist-test-slave-h6n0"
server_key: "9759c37f915b17cb9aa4721a39190179"
server_key_iv: "2df100577c38585795cb31edc0d3410f"
server_key_version: "encryptionkey@0"
I20260811 19:02:54.135759 30229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/minicluster-data/master-0/wal/instance:
uuid: "a0c12cf38d684de287a83812a1bd4d32"
format_stamp: "Formatted at 2026-08-11 19:02:54 on dist-test-slave-h6n0"
server_key: "9759c37f915b17cb9aa4721a39190179"
server_key_iv: "2df100577c38585795cb31edc0d3410f"
server_key_version: "encryptionkey@0"
I20260811 19:02:54.145613 30229 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.009s sys 0.000s
I20260811 19:02:54.150365 30251 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:54.153182 30229 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260811 19:02:54.153427 30229 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/minicluster-data/master-0/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/minicluster-data/master-0/wal
uuid: "a0c12cf38d684de287a83812a1bd4d32"
format_stamp: "Formatted at 2026-08-11 19:02:54 on dist-test-slave-h6n0"
server_key: "9759c37f915b17cb9aa4721a39190179"
server_key_iv: "2df100577c38585795cb31edc0d3410f"
server_key_version: "encryptionkey@0"
I20260811 19:02:54.153956 30229 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:02:54.183925 30229 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:02:54.189515 30229 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:02:54.189813 30229 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:02:54.218071 30229 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.190:36845
I20260811 19:02:54.218144 30353 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.190:36845 every 8 connection(s)
I20260811 19:02:54.220136 30229 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
I20260811 19:02:54.222815 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 30229
I20260811 19:02:54.223039 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/minicluster-data/master-0/wal/instance
I20260811 19:02:54.223573 30146 external_mini_cluster.cc:1428] Setting key bd73e955bb713de1b08e583013332b53
I20260811 19:02:54.225780 30354 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 19:02:54 dist-test-slave-h6n0 krb5kdc[30207](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474973, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 19:02:54.247084 30354 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0c12cf38d684de287a83812a1bd4d32: Bootstrap starting.
I20260811 19:02:54.250725 30354 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a0c12cf38d684de287a83812a1bd4d32: Neither blocks nor log segments found. Creating new log.
I20260811 19:02:54.251919 30354 log.cc:824] T 00000000000000000000000000000000 P a0c12cf38d684de287a83812a1bd4d32: Log is configured to *not* fsync() on all Append() calls
I20260811 19:02:54.255143 30354 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0c12cf38d684de287a83812a1bd4d32: No bootstrap required, opened a new log
I20260811 19:02:54.258123 30359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:54.226254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58116 (local address 127.29.112.190:36845)
0811 19:02:54.227593 (+ 1339us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:54.227613 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:54.228220 (+ 607us) server_negotiation.cc:407] Connection header received
0811 19:02:54.231310 (+ 3090us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:54.231350 (+ 40us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:54.231976 (+ 626us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:54.232516 (+ 540us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:02:54.235378 (+ 2862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:54.237248 (+ 1870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:54.239904 (+ 2656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:54.241008 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:54.241480 (+ 472us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:54.248420 (+ 6940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:02:54.248432 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:02:54.248451 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:02:54.248510 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:02:54.252423 (+ 3913us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:54.253332 (+ 909us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:54.253341 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:54.253348 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:54.253547 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:54.254028 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:54.254033 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:54.254035 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:54.254708 (+ 673us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:02:54.255089 (+ 381us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:54.255594 (+ 505us) server_negotiation.cc:299] Negotiation successful
0811 19:02:54.256104 (+ 510us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":978,"thread_start_us":618,"threads_started":1}
I20260811 19:02:54.266937 30354 raft_consensus.cc:348] T 00000000000000000000000000000000 P a0c12cf38d684de287a83812a1bd4d32 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0c12cf38d684de287a83812a1bd4d32" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 36845 } }
I20260811 19:02:54.267597 30354 raft_consensus.cc:374] T 00000000000000000000000000000000 P a0c12cf38d684de287a83812a1bd4d32 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:02:54.267733 30354 raft_consensus.cc:729] T 00000000000000000000000000000000 P a0c12cf38d684de287a83812a1bd4d32 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0c12cf38d684de287a83812a1bd4d32, State: Initialized, Role: FOLLOWER
I20260811 19:02:54.268474 30354 consensus_queue.cc:260] T 00000000000000000000000000000000 P a0c12cf38d684de287a83812a1bd4d32 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0c12cf38d684de287a83812a1bd4d32" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 36845 } }
I20260811 19:02:54.268770 30354 raft_consensus.cc:388] T 00000000000000000000000000000000 P a0c12cf38d684de287a83812a1bd4d32 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:02:54.268952 30354 raft_consensus.cc:482] T 00000000000000000000000000000000 P a0c12cf38d684de287a83812a1bd4d32 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:02:54.269156 30354 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a0c12cf38d684de287a83812a1bd4d32 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:02:54.271852 30354 raft_consensus.cc:504] T 00000000000000000000000000000000 P a0c12cf38d684de287a83812a1bd4d32 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0c12cf38d684de287a83812a1bd4d32" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 36845 } }
I20260811 19:02:54.272567 30354 leader_election.cc:302] T 00000000000000000000000000000000 P a0c12cf38d684de287a83812a1bd4d32 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a0c12cf38d684de287a83812a1bd4d32; no voters:
I20260811 19:02:54.273325 30354 leader_election.cc:288] T 00000000000000000000000000000000 P a0c12cf38d684de287a83812a1bd4d32 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:02:54.274327 30363 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a0c12cf38d684de287a83812a1bd4d32 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:02:54.275928 30354 sys_catalog.cc:564] T 00000000000000000000000000000000 P a0c12cf38d684de287a83812a1bd4d32 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:02:54.278383 30363 raft_consensus.cc:686] T 00000000000000000000000000000000 P a0c12cf38d684de287a83812a1bd4d32 [term 1 LEADER]: Becoming Leader. State: Replica: a0c12cf38d684de287a83812a1bd4d32, State: Running, Role: LEADER
I20260811 19:02:54.281646 30363 consensus_queue.cc:237] T 00000000000000000000000000000000 P a0c12cf38d684de287a83812a1bd4d32 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0c12cf38d684de287a83812a1bd4d32" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 36845 } }
I20260811 19:02:54.287384 30364 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0c12cf38d684de287a83812a1bd4d32 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a0c12cf38d684de287a83812a1bd4d32" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0c12cf38d684de287a83812a1bd4d32" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 36845 } } }
I20260811 19:02:54.287962 30364 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0c12cf38d684de287a83812a1bd4d32 [sys.catalog]: This master's current role is: LEADER
I20260811 19:02:54.291447 30390 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:02:54.290716 30365 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0c12cf38d684de287a83812a1bd4d32 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a0c12cf38d684de287a83812a1bd4d32. Latest consensus state: current_term: 1 leader_uuid: "a0c12cf38d684de287a83812a1bd4d32" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0c12cf38d684de287a83812a1bd4d32" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 36845 } } }
I20260811 19:02:54.291973 30365 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0c12cf38d684de287a83812a1bd4d32 [sys.catalog]: This master's current role is: LEADER
I20260811 19:02:54.298592 30390 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:02:54.309500 30390 catalog_manager.cc:1347] Generated new cluster ID: 59d85ee539f540cbbe4ef40c2ec70985
I20260811 19:02:54.309644 30390 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:02:54.325728 30390 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:02:54.328001 30390 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:02:54.337051 30390 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a0c12cf38d684de287a83812a1bd4d32: Generated new TSK 0
I20260811 19:02:54.338465 30390 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.112.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.129.keytab.
Entry for principal kudu/127.29.112.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.129.keytab.
I20260811 19:02:54.428614 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.129:0
--local_ip_for_outbound_sockets=127.29.112.129
--webserver_interface=127.29.112.129
--webserver_port=0
--tserver_master_addrs=127.29.112.190:36845
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.129.keytab
--principal=kudu/127.29.112.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:39161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:02:54.853984 30402 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:02:54.854346 30402 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:02:54.854457 30402 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:02:54.863837 30402 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:02:54.863986 30402 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:02:54.864107 30402 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.129
I20260811 19:02:54.875995 30402 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:39161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.129.keytab
--principal=kudu/127.29.112.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.112.129
--webserver_port=0
--tserver_master_addrs=127.29.112.190:36845
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.129
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:02:54.877633 30402 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:02:54.879081 30402 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:02:54.894307 30408 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:02:54.894460 30409 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:02:54.894695 30412 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:02:54.896095 30402 server_base.cc:1034] running on GCE node
I20260811 19:02:54.896939 30402 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:02:54.898142 30402 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:02:54.899405 30402 hybrid_clock.cc:648] HybridClock initialized: now 1786474974899325 us; error 58 us; skew 500 ppm
I20260811 19:02:54.899811 30402 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:02:54 dist-test-slave-h6n0 krb5kdc[30207](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474974, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:02:54.903460 30402 init.cc:377] Logged in from keytab as kudu/127.29.112.129@KRBTEST.COM (short username kudu)
I20260811 19:02:54.905184 30402 webserver.cc:460] Webserver started at http://127.29.112.129:42289/ using document root <none> and password file <none>
I20260811 19:02:54.905941 30402 fs_manager.cc:359] Metadata directory not provided
I20260811 19:02:54.906049 30402 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:02:54.906365 30402 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:02:54.909456 30402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/minicluster-data/ts-0/data/instance:
uuid: "2bd887277b2d4ff9a177548a7e23b585"
format_stamp: "Formatted at 2026-08-11 19:02:54 on dist-test-slave-h6n0"
server_key: "a1c996208c98adf3b9522b60be7635ea"
server_key_iv: "e02c865a2e0a0b92b6b527a784fe10bd"
server_key_version: "encryptionkey@0"
I20260811 19:02:54.910285 30402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/minicluster-data/ts-0/wal/instance:
uuid: "2bd887277b2d4ff9a177548a7e23b585"
format_stamp: "Formatted at 2026-08-11 19:02:54 on dist-test-slave-h6n0"
server_key: "a1c996208c98adf3b9522b60be7635ea"
server_key_iv: "e02c865a2e0a0b92b6b527a784fe10bd"
server_key_version: "encryptionkey@0"
I20260811 19:02:54.916815 30402 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 19:02:54.920728 30426 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:54.922956 30402 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 19:02:54.923306 30402 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/minicluster-data/ts-0/wal
uuid: "2bd887277b2d4ff9a177548a7e23b585"
format_stamp: "Formatted at 2026-08-11 19:02:54 on dist-test-slave-h6n0"
server_key: "a1c996208c98adf3b9522b60be7635ea"
server_key_iv: "e02c865a2e0a0b92b6b527a784fe10bd"
server_key_version: "encryptionkey@0"
I20260811 19:02:54.923657 30402 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:02:54.938827 30402 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:02:54.944236 30402 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:02:54.944590 30402 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:02:54.945746 30402 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:02:54.947327 30402 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:02:54.947439 30402 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:54.947609 30402 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:02:54.947686 30402 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:54.996444 30402 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.129:34731
I20260811 19:02:54.996490 30624 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.129:34731 every 8 connection(s)
I20260811 19:02:54.998657 30402 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/minicluster-data/ts-0/data/info.pb
I20260811 19:02:54.999987 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 30402
I20260811 19:02:55.000291 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/minicluster-data/ts-0/wal/instance
I20260811 19:02:55.000947 30146 external_mini_cluster.cc:1428] Setting key 8be3bc0aa6b287d99378014a945c1fc0
Aug 11 19:02:55 dist-test-slave-h6n0 krb5kdc[30207](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474974, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.129@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 19:02:55.028354 30628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:55.003137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.129:43599 (local address 127.29.112.190:36845)
0811 19:02:55.003864 (+ 727us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:55.003872 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:55.005545 (+ 1673us) server_negotiation.cc:407] Connection header received
0811 19:02:55.007787 (+ 2242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:55.007796 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:55.007898 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:55.008027 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:02:55.009873 (+ 1846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:55.011104 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:55.013189 (+ 2085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:55.014312 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:55.014499 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:55.019599 (+ 5100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:02:55.019610 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:02:55.019624 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:02:55.019691 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:02:55.023682 (+ 3991us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:55.024901 (+ 1219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:55.024910 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:55.024916 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:55.025133 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:55.025772 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:55.025781 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:55.025787 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:55.026248 (+ 461us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:02:55.026420 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:55.027526 (+ 1106us) server_negotiation.cc:299] Negotiation successful
0811 19:02:55.027820 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":591,"thread_start_us":317,"threads_started":1}
I20260811 19:02:55.029093 30630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:55.003978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:36845 (local address 127.29.112.129:43599)
0811 19:02:55.005324 (+ 1346us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:55.005383 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 19:02:55.007483 (+ 2100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:02:55.008341 (+ 858us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:02:55.008351 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:02:55.009005 (+ 654us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:02:55.009671 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:55.009690 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:55.011309 (+ 1619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:55.011314 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:55.012811 (+ 1497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:55.012825 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:55.014684 (+ 1859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:55.014693 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:55.014925 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:55.015815 (+ 890us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:02:55.015843 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:02:55.019166 (+ 3323us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:02:55.023951 (+ 4785us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:02:55.023963 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:02:55.023984 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:02:55.024696 (+ 712us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:02:55.025326 (+ 630us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:02:55.025332 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:02:55.025337 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:02:55.025610 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:02:55.026430 (+ 820us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:02:55.026437 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:02:55.027054 (+ 617us) client_negotiation.cc:769] Sending connection context
0811 19:02:55.027372 (+ 318us) client_negotiation.cc:240] Negotiation successful
0811 19:02:55.027835 (+ 463us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1019,"thread_start_us":717,"threads_started":1}
I20260811 19:02:55.031069 30625 heartbeater.cc:347] Connected to a master server at 127.29.112.190:36845
I20260811 19:02:55.031419 30625 heartbeater.cc:464] Registering TS with master...
I20260811 19:02:55.032207 30625 heartbeater.cc:511] Master 127.29.112.190:36845 requested a full tablet report, sending...
I20260811 19:02:55.034876 30288 ts_manager.cc:194] Registered new tserver with Master: 2bd887277b2d4ff9a177548a7e23b585 (127.29.112.129:34731)
I20260811 19:02:55.036392 30288 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.129@KRBTEST.COM'} at 127.29.112.129:43599
WARNING: no policy specified for kudu/127.29.112.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.130.keytab.
Entry for principal kudu/127.29.112.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.130.keytab.
I20260811 19:02:55.091181 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.130:0
--local_ip_for_outbound_sockets=127.29.112.130
--webserver_interface=127.29.112.130
--webserver_port=0
--tserver_master_addrs=127.29.112.190:36845
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.130.keytab
--principal=kudu/127.29.112.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:39161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:02:55.514398 30634 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:02:55.514775 30634 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:02:55.514881 30634 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:02:55.524181 30634 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:02:55.524385 30634 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:02:55.524544 30634 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.130
I20260811 19:02:55.537626 30634 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:39161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.130.keytab
--principal=kudu/127.29.112.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.112.130
--webserver_port=0
--tserver_master_addrs=127.29.112.190:36845
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.130
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:02:55.539737 30634 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:02:55.541175 30634 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:02:55.558202 30648 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:02:55.558261 30640 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:02:55.558295 30643 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:02:55.559476 30634 server_base.cc:1034] running on GCE node
I20260811 19:02:55.560262 30634 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:02:55.561461 30634 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:02:55.562676 30634 hybrid_clock.cc:648] HybridClock initialized: now 1786474975562614 us; error 48 us; skew 500 ppm
I20260811 19:02:55.563064 30634 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:02:55 dist-test-slave-h6n0 krb5kdc[30207](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474975, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:02:55.567076 30634 init.cc:377] Logged in from keytab as kudu/127.29.112.130@KRBTEST.COM (short username kudu)
I20260811 19:02:55.569245 30634 webserver.cc:460] Webserver started at http://127.29.112.130:46525/ using document root <none> and password file <none>
I20260811 19:02:55.570045 30634 fs_manager.cc:359] Metadata directory not provided
I20260811 19:02:55.570217 30634 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:02:55.570653 30634 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:02:55.573992 30634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/minicluster-data/ts-1/data/instance:
uuid: "d3d191ec09604e43899c8b0620000338"
format_stamp: "Formatted at 2026-08-11 19:02:55 on dist-test-slave-h6n0"
server_key: "cec0c6522d348f8f6312c33c9055b568"
server_key_iv: "a2e5ed980e1b275a8862c6abdf2aef5b"
server_key_version: "encryptionkey@0"
I20260811 19:02:55.574959 30634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/minicluster-data/ts-1/wal/instance:
uuid: "d3d191ec09604e43899c8b0620000338"
format_stamp: "Formatted at 2026-08-11 19:02:55 on dist-test-slave-h6n0"
server_key: "cec0c6522d348f8f6312c33c9055b568"
server_key_iv: "a2e5ed980e1b275a8862c6abdf2aef5b"
server_key_version: "encryptionkey@0"
I20260811 19:02:55.581980 30634 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.001s
I20260811 19:02:55.587646 30661 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:55.589735 30634 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260811 19:02:55.590011 30634 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/minicluster-data/ts-1/wal
uuid: "d3d191ec09604e43899c8b0620000338"
format_stamp: "Formatted at 2026-08-11 19:02:55 on dist-test-slave-h6n0"
server_key: "cec0c6522d348f8f6312c33c9055b568"
server_key_iv: "a2e5ed980e1b275a8862c6abdf2aef5b"
server_key_version: "encryptionkey@0"
I20260811 19:02:55.590420 30634 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:02:55.610697 30634 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:02:55.617744 30634 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:02:55.618166 30634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:02:55.619318 30634 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:02:55.621364 30634 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:02:55.621502 30634 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:55.621722 30634 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:02:55.621812 30634 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:55.665820 30634 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.130:40247
I20260811 19:02:55.665864 30866 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.130:40247 every 8 connection(s)
I20260811 19:02:55.667356 30634 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/minicluster-data/ts-1/data/info.pb
I20260811 19:02:55.673540 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 30634
I20260811 19:02:55.673774 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/minicluster-data/ts-1/wal/instance
I20260811 19:02:55.674472 30146 external_mini_cluster.cc:1428] Setting key e4eaec78071ea5a54938e916ba7f9f42
Aug 11 19:02:55 dist-test-slave-h6n0 krb5kdc[30207](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474975, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.130@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 19:02:55.699278 30871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:55.670786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.130:44455 (local address 127.29.112.190:36845)
0811 19:02:55.674977 (+ 4191us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:55.674985 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:55.675782 (+ 797us) server_negotiation.cc:407] Connection header received
0811 19:02:55.678823 (+ 3041us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:55.678829 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:55.678933 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:55.679080 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:02:55.681566 (+ 2486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:55.682396 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:55.685264 (+ 2868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:55.685869 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:55.686031 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:55.691141 (+ 5110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:02:55.691148 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:02:55.691152 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:02:55.691196 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:02:55.695542 (+ 4346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:55.696574 (+ 1032us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:55.696586 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:55.696588 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:55.696728 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:55.697248 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:55.697253 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:55.697255 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:55.697576 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:02:55.697756 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:55.698580 (+ 824us) server_negotiation.cc:299] Negotiation successful
0811 19:02:55.698849 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4037,"thread_start_us":3914,"threads_started":1}
I20260811 19:02:55.700445 30872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:55.671367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:36845 (local address 127.29.112.130:44455)
0811 19:02:55.675583 (+ 4216us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:55.675641 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 19:02:55.678359 (+ 2718us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:02:55.679429 (+ 1070us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:02:55.679446 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:02:55.680385 (+ 939us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:02:55.681303 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:55.681346 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:55.682612 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:55.682620 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:55.684988 (+ 2368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:55.685009 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:55.686210 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:55.686221 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:55.686691 (+ 470us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:55.687617 (+ 926us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:02:55.687655 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:02:55.690842 (+ 3187us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:02:55.695734 (+ 4892us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:02:55.695752 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:02:55.695766 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:02:55.696409 (+ 643us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:02:55.696859 (+ 450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:02:55.696867 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:02:55.696873 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:02:55.697100 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:02:55.697724 (+ 624us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:02:55.697732 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:02:55.698126 (+ 394us) client_negotiation.cc:769] Sending connection context
0811 19:02:55.698432 (+ 306us) client_negotiation.cc:240] Negotiation successful
0811 19:02:55.699067 (+ 635us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3869,"thread_start_us":3625,"threads_started":1}
I20260811 19:02:55.702694 30867 heartbeater.cc:347] Connected to a master server at 127.29.112.190:36845
I20260811 19:02:55.703086 30867 heartbeater.cc:464] Registering TS with master...
I20260811 19:02:55.703955 30867 heartbeater.cc:511] Master 127.29.112.190:36845 requested a full tablet report, sending...
I20260811 19:02:55.705963 30288 ts_manager.cc:194] Registered new tserver with Master: d3d191ec09604e43899c8b0620000338 (127.29.112.130:40247)
I20260811 19:02:55.707340 30288 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.130@KRBTEST.COM'} at 127.29.112.130:44455
WARNING: no policy specified for kudu/127.29.112.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.131.keytab.
Entry for principal kudu/127.29.112.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.131.keytab.
I20260811 19:02:55.776811 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.131:0
--local_ip_for_outbound_sockets=127.29.112.131
--webserver_interface=127.29.112.131
--webserver_port=0
--tserver_master_addrs=127.29.112.190:36845
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.131.keytab
--principal=kudu/127.29.112.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:39161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 19:02:56.039777 30625 heartbeater.cc:503] Master 127.29.112.190:36845 was elected leader, sending a full tablet report...
W20260811 19:02:56.206751 30879 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:02:56.207114 30879 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:02:56.207284 30879 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:02:56.215633 30879 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:02:56.215826 30879 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:02:56.215999 30879 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.131
I20260811 19:02:56.227916 30879 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:39161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.131.keytab
--principal=kudu/127.29.112.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.112.131
--webserver_port=0
--tserver_master_addrs=127.29.112.190:36845
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.131
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:02:56.229393 30879 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:02:56.230859 30879 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:02:56.248087 30890 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:02:56.248206 30892 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:02:56.248116 30894 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:02:56.251019 30879 server_base.cc:1034] running on GCE node
I20260811 19:02:56.251842 30879 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:02:56.253106 30879 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:02:56.256609 30879 hybrid_clock.cc:648] HybridClock initialized: now 1786474976256433 us; error 162 us; skew 500 ppm
I20260811 19:02:56.257612 30879 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:02:56 dist-test-slave-h6n0 krb5kdc[30207](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474976, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:02:56.263430 30879 init.cc:377] Logged in from keytab as kudu/127.29.112.131@KRBTEST.COM (short username kudu)
I20260811 19:02:56.265626 30879 webserver.cc:460] Webserver started at http://127.29.112.131:37369/ using document root <none> and password file <none>
I20260811 19:02:56.266664 30879 fs_manager.cc:359] Metadata directory not provided
I20260811 19:02:56.266847 30879 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:02:56.267287 30879 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:02:56.271801 30879 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/minicluster-data/ts-2/data/instance:
uuid: "1ab526b793bd471390337d39a0097c47"
format_stamp: "Formatted at 2026-08-11 19:02:56 on dist-test-slave-h6n0"
server_key: "7fdc117298ce349653e972602c29619c"
server_key_iv: "a4f4a52500baa2cfaf52111a2f942dee"
server_key_version: "encryptionkey@0"
I20260811 19:02:56.273262 30879 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/minicluster-data/ts-2/wal/instance:
uuid: "1ab526b793bd471390337d39a0097c47"
format_stamp: "Formatted at 2026-08-11 19:02:56 on dist-test-slave-h6n0"
server_key: "7fdc117298ce349653e972602c29619c"
server_key_iv: "a4f4a52500baa2cfaf52111a2f942dee"
server_key_version: "encryptionkey@0"
I20260811 19:02:56.281450 30879 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.009s sys 0.000s
I20260811 19:02:56.285548 30910 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:56.287283 30879 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 19:02:56.287463 30879 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/minicluster-data/ts-2/wal
uuid: "1ab526b793bd471390337d39a0097c47"
format_stamp: "Formatted at 2026-08-11 19:02:56 on dist-test-slave-h6n0"
server_key: "7fdc117298ce349653e972602c29619c"
server_key_iv: "a4f4a52500baa2cfaf52111a2f942dee"
server_key_version: "encryptionkey@0"
I20260811 19:02:56.287690 30879 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:02:56.318907 30879 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:02:56.325043 30879 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:02:56.325404 30879 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:02:56.326869 30879 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:02:56.328509 30879 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:02:56.328637 30879 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:56.328750 30879 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:02:56.328799 30879 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:56.373382 30879 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.131:35975
I20260811 19:02:56.373428 31106 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.131:35975 every 8 connection(s)
I20260811 19:02:56.374920 30879 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/minicluster-data/ts-2/data/info.pb
I20260811 19:02:56.381750 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 30879
I20260811 19:02:56.381978 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/minicluster-data/ts-2/wal/instance
I20260811 19:02:56.382460 30146 external_mini_cluster.cc:1428] Setting key 55f63b58b2e41ebc79c3584a06034bb6
Aug 11 19:02:56 dist-test-slave-h6n0 krb5kdc[30207](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474976, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.131@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 19:02:56.402922 31111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:56.378823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.131:43707 (local address 127.29.112.190:36845)
0811 19:02:56.379630 (+ 807us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:56.379636 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:56.380708 (+ 1072us) server_negotiation.cc:407] Connection header received
0811 19:02:56.383675 (+ 2967us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:56.383682 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:56.383898 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:56.384167 (+ 269us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:02:56.386689 (+ 2522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:56.387581 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:56.389218 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:56.389630 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:56.389776 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:56.394447 (+ 4671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:02:56.394452 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:02:56.394456 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:02:56.394495 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:02:56.399063 (+ 4568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:56.400126 (+ 1063us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:56.400131 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:56.400134 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:56.400285 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:56.400784 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:56.400788 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:56.400791 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:56.401118 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:02:56.401252 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:56.402256 (+ 1004us) server_negotiation.cc:299] Negotiation successful
0811 19:02:56.402462 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":663,"thread_start_us":459,"threads_started":1}
I20260811 19:02:56.404426 31112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:56.379560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:36845 (local address 127.29.112.131:43707)
0811 19:02:56.380496 (+ 936us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:56.380569 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0811 19:02:56.383156 (+ 2587us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:02:56.384673 (+ 1517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:02:56.384686 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:02:56.385598 (+ 912us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:02:56.386432 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:56.386457 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:56.387757 (+ 1300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:56.387762 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:56.389053 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:56.389073 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:56.389876 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:56.389887 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:56.390135 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:56.390949 (+ 814us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:02:56.390973 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:02:56.393993 (+ 3020us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:02:56.399368 (+ 5375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:02:56.399380 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:02:56.399399 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:02:56.399915 (+ 516us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:02:56.400425 (+ 510us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:02:56.400429 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:02:56.400432 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:02:56.400647 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:02:56.401260 (+ 613us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:02:56.401267 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:02:56.401631 (+ 364us) client_negotiation.cc:769] Sending connection context
0811 19:02:56.402171 (+ 540us) client_negotiation.cc:240] Negotiation successful
0811 19:02:56.403017 (+ 846us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":659,"thread_start_us":373,"threads_started":1}
I20260811 19:02:56.405921 31107 heartbeater.cc:347] Connected to a master server at 127.29.112.190:36845
I20260811 19:02:56.406260 31107 heartbeater.cc:464] Registering TS with master...
I20260811 19:02:56.407063 31107 heartbeater.cc:511] Master 127.29.112.190:36845 requested a full tablet report, sending...
I20260811 19:02:56.408703 30288 ts_manager.cc:194] Registered new tserver with Master: 1ab526b793bd471390337d39a0097c47 (127.29.112.131:35975)
I20260811 19:02:56.409700 30288 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.112.131@KRBTEST.COM'} at 127.29.112.131:43707
I20260811 19:02:56.412081 30146 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 19:02:56.433736 31111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:56.418244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58128 (local address 127.29.112.190:36845)
0811 19:02:56.418460 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:56.418465 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:56.419180 (+ 715us) server_negotiation.cc:407] Connection header received
0811 19:02:56.419288 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:56.419292 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:56.419362 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:56.419466 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:02:56.421008 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:56.421782 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:56.423234 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:56.423716 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:56.423837 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:56.427162 (+ 3325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:02:56.427169 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:02:56.427173 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:02:56.427207 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:02:56.430430 (+ 3223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:56.431256 (+ 826us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:56.431261 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:56.431268 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:56.431421 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:56.431947 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:56.431954 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:56.431957 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:56.432555 (+ 598us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:02:56.432802 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:56.433288 (+ 486us) server_negotiation.cc:299] Negotiation successful
0811 19:02:56.433488 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260811 19:02:56.440865 30288 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58128:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 19:02:56.449334 30288 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 19:02:56.476862 31131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:56.464908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.130:40247 (local address 127.0.0.1:47398)
0811 19:02:56.465766 (+ 858us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:56.465814 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 19:02:56.465986 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:02:56.467740 (+ 1754us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:02:56.467752 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:02:56.467792 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:02:56.468225 (+ 433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:56.468237 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:56.469986 (+ 1749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:56.469997 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:56.473187 (+ 3190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:56.473207 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:56.475517 (+ 2310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:56.475528 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:56.475815 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:56.475880 (+ 65us) client_negotiation.cc:769] Sending connection context
0811 19:02:56.476016 (+ 136us) client_negotiation.cc:240] Negotiation successful
0811 19:02:56.476260 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":589,"mutex_wait_us":1,"thread_start_us":403,"threads_started":1}
I20260811 19:02:56.477087 31132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:56.465104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47398 (local address 127.29.112.130:40247)
0811 19:02:56.467005 (+ 1901us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:56.467016 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:56.467049 (+ 33us) server_negotiation.cc:407] Connection header received
0811 19:02:56.467168 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:56.467179 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:56.467450 (+ 271us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:56.468423 (+ 973us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:02:56.468613 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:56.469780 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:56.473392 (+ 3612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:56.475163 (+ 1771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:56.476244 (+ 1081us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:56.476460 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:56.476544 (+ 84us) server_negotiation.cc:299] Negotiation successful
0811 19:02:56.476713 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1677,"thread_start_us":1482,"threads_started":1}
I20260811 19:02:56.478269 31136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:56.465529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.131:35975 (local address 127.0.0.1:59240)
0811 19:02:56.467339 (+ 1810us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:56.467365 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 19:02:56.467522 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:02:56.469728 (+ 2206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:02:56.469734 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:02:56.469770 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:02:56.470622 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:56.470640 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:56.472707 (+ 2067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:56.472715 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:56.475103 (+ 2388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:56.475125 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:56.477440 (+ 2315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:56.477446 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:56.477624 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:56.477658 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 19:02:56.477730 (+ 72us) client_negotiation.cc:240] Negotiation successful
0811 19:02:56.477867 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1604,"mutex_wait_us":1,"thread_start_us":4874,"threads_started":1}
I20260811 19:02:56.479000 31137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:56.466351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59240 (local address 127.29.112.131:35975)
0811 19:02:56.468900 (+ 2549us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:56.468912 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:56.468957 (+ 45us) server_negotiation.cc:407] Connection header received
0811 19:02:56.469081 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:56.469093 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:56.469441 (+ 348us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:56.469715 (+ 274us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:02:56.470896 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:56.472555 (+ 1659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:56.475327 (+ 2772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:56.477182 (+ 1855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:56.477626 (+ 444us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:56.477940 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:56.478101 (+ 161us) server_negotiation.cc:299] Negotiation successful
0811 19:02:56.478503 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2330,"thread_start_us":2139,"threads_started":1}
I20260811 19:02:56.479780 31133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:56.466235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:34731 (local address 127.0.0.1:32834)
0811 19:02:56.471056 (+ 4821us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:56.471082 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 19:02:56.471226 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:02:56.472048 (+ 822us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:02:56.472054 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:02:56.472082 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:02:56.472521 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:56.472533 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:56.474271 (+ 1738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:56.474277 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:56.476670 (+ 2393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:56.476685 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:56.478937 (+ 2252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:56.478944 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:56.479184 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:56.479208 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 19:02:56.479335 (+ 127us) client_negotiation.cc:240] Negotiation successful
0811 19:02:56.479481 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4639,"thread_start_us":754,"threads_started":1}
I20260811 19:02:56.480857 31138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:56.467107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32834 (local address 127.29.112.129:34731)
0811 19:02:56.468302 (+ 1195us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:56.468313 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:56.471368 (+ 3055us) server_negotiation.cc:407] Connection header received
0811 19:02:56.471484 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:56.471492 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:56.471828 (+ 336us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:56.472036 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:02:56.472760 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:56.474105 (+ 1345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:56.476950 (+ 2845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:56.478638 (+ 1688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:56.478870 (+ 232us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:56.479054 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:56.480333 (+ 1279us) server_negotiation.cc:299] Negotiation successful
0811 19:02:56.480518 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":993,"thread_start_us":807,"threads_started":1}
I20260811 19:02:56.482705 30987 tablet_service.cc:1467] Processing CreateTablet for tablet 1fcda57fe38545ae9f7bcc150fb6b95b (DEFAULT_TABLE table=test-table [id=9f2b6b3bb42a4b98b1b8f722b9e47de9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:02:56.482705 30753 tablet_service.cc:1467] Processing CreateTablet for tablet 1fcda57fe38545ae9f7bcc150fb6b95b (DEFAULT_TABLE table=test-table [id=9f2b6b3bb42a4b98b1b8f722b9e47de9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:02:56.483839 30506 tablet_service.cc:1467] Processing CreateTablet for tablet 1fcda57fe38545ae9f7bcc150fb6b95b (DEFAULT_TABLE table=test-table [id=9f2b6b3bb42a4b98b1b8f722b9e47de9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:02:56.484251 30987 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1fcda57fe38545ae9f7bcc150fb6b95b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:02:56.484256 30753 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1fcda57fe38545ae9f7bcc150fb6b95b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:02:56.484951 30506 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1fcda57fe38545ae9f7bcc150fb6b95b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:02:56.499922 31140 tablet_bootstrap.cc:492] T 1fcda57fe38545ae9f7bcc150fb6b95b P 1ab526b793bd471390337d39a0097c47: Bootstrap starting.
I20260811 19:02:56.502146 31139 tablet_bootstrap.cc:492] T 1fcda57fe38545ae9f7bcc150fb6b95b P 2bd887277b2d4ff9a177548a7e23b585: Bootstrap starting.
I20260811 19:02:56.503207 31141 tablet_bootstrap.cc:492] T 1fcda57fe38545ae9f7bcc150fb6b95b P d3d191ec09604e43899c8b0620000338: Bootstrap starting.
I20260811 19:02:56.505218 31140 tablet_bootstrap.cc:654] T 1fcda57fe38545ae9f7bcc150fb6b95b P 1ab526b793bd471390337d39a0097c47: Neither blocks nor log segments found. Creating new log.
I20260811 19:02:56.506275 31141 tablet_bootstrap.cc:654] T 1fcda57fe38545ae9f7bcc150fb6b95b P d3d191ec09604e43899c8b0620000338: Neither blocks nor log segments found. Creating new log.
I20260811 19:02:56.506894 31140 log.cc:824] T 1fcda57fe38545ae9f7bcc150fb6b95b P 1ab526b793bd471390337d39a0097c47: Log is configured to *not* fsync() on all Append() calls
I20260811 19:02:56.507472 31141 log.cc:824] T 1fcda57fe38545ae9f7bcc150fb6b95b P d3d191ec09604e43899c8b0620000338: Log is configured to *not* fsync() on all Append() calls
I20260811 19:02:56.508867 31139 tablet_bootstrap.cc:654] T 1fcda57fe38545ae9f7bcc150fb6b95b P 2bd887277b2d4ff9a177548a7e23b585: Neither blocks nor log segments found. Creating new log.
I20260811 19:02:56.510342 31139 log.cc:824] T 1fcda57fe38545ae9f7bcc150fb6b95b P 2bd887277b2d4ff9a177548a7e23b585: Log is configured to *not* fsync() on all Append() calls
I20260811 19:02:56.511296 31140 tablet_bootstrap.cc:492] T 1fcda57fe38545ae9f7bcc150fb6b95b P 1ab526b793bd471390337d39a0097c47: No bootstrap required, opened a new log
I20260811 19:02:56.511639 31140 ts_tablet_manager.cc:1397] T 1fcda57fe38545ae9f7bcc150fb6b95b P 1ab526b793bd471390337d39a0097c47: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.010s
I20260811 19:02:56.514658 31139 tablet_bootstrap.cc:492] T 1fcda57fe38545ae9f7bcc150fb6b95b P 2bd887277b2d4ff9a177548a7e23b585: No bootstrap required, opened a new log
I20260811 19:02:56.515163 31141 tablet_bootstrap.cc:492] T 1fcda57fe38545ae9f7bcc150fb6b95b P d3d191ec09604e43899c8b0620000338: No bootstrap required, opened a new log
I20260811 19:02:56.515399 31139 ts_tablet_manager.cc:1397] T 1fcda57fe38545ae9f7bcc150fb6b95b P 2bd887277b2d4ff9a177548a7e23b585: Time spent bootstrapping tablet: real 0.014s user 0.005s sys 0.005s
I20260811 19:02:56.515467 31141 ts_tablet_manager.cc:1397] T 1fcda57fe38545ae9f7bcc150fb6b95b P d3d191ec09604e43899c8b0620000338: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.006s
I20260811 19:02:56.520128 31140 raft_consensus.cc:348] T 1fcda57fe38545ae9f7bcc150fb6b95b P 1ab526b793bd471390337d39a0097c47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3d191ec09604e43899c8b0620000338" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 40247 } } peers { permanent_uuid: "2bd887277b2d4ff9a177548a7e23b585" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 34731 } } peers { permanent_uuid: "1ab526b793bd471390337d39a0097c47" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 35975 } }
I20260811 19:02:56.520546 31140 raft_consensus.cc:374] T 1fcda57fe38545ae9f7bcc150fb6b95b P 1ab526b793bd471390337d39a0097c47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:02:56.520642 31140 raft_consensus.cc:729] T 1fcda57fe38545ae9f7bcc150fb6b95b P 1ab526b793bd471390337d39a0097c47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ab526b793bd471390337d39a0097c47, State: Initialized, Role: FOLLOWER
I20260811 19:02:56.521317 31140 consensus_queue.cc:260] T 1fcda57fe38545ae9f7bcc150fb6b95b P 1ab526b793bd471390337d39a0097c47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3d191ec09604e43899c8b0620000338" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 40247 } } peers { permanent_uuid: "2bd887277b2d4ff9a177548a7e23b585" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 34731 } } peers { permanent_uuid: "1ab526b793bd471390337d39a0097c47" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 35975 } }
I20260811 19:02:56.522447 31107 heartbeater.cc:503] Master 127.29.112.190:36845 was elected leader, sending a full tablet report...
I20260811 19:02:56.523164 31140 ts_tablet_manager.cc:1428] T 1fcda57fe38545ae9f7bcc150fb6b95b P 1ab526b793bd471390337d39a0097c47: Time spent starting tablet: real 0.011s user 0.009s sys 0.003s
I20260811 19:02:56.528957 31141 raft_consensus.cc:348] T 1fcda57fe38545ae9f7bcc150fb6b95b P d3d191ec09604e43899c8b0620000338 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3d191ec09604e43899c8b0620000338" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 40247 } } peers { permanent_uuid: "2bd887277b2d4ff9a177548a7e23b585" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 34731 } } peers { permanent_uuid: "1ab526b793bd471390337d39a0097c47" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 35975 } }
I20260811 19:02:56.528949 31139 raft_consensus.cc:348] T 1fcda57fe38545ae9f7bcc150fb6b95b P 2bd887277b2d4ff9a177548a7e23b585 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3d191ec09604e43899c8b0620000338" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 40247 } } peers { permanent_uuid: "2bd887277b2d4ff9a177548a7e23b585" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 34731 } } peers { permanent_uuid: "1ab526b793bd471390337d39a0097c47" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 35975 } }
I20260811 19:02:56.529462 31139 raft_consensus.cc:374] T 1fcda57fe38545ae9f7bcc150fb6b95b P 2bd887277b2d4ff9a177548a7e23b585 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:02:56.529462 31141 raft_consensus.cc:374] T 1fcda57fe38545ae9f7bcc150fb6b95b P d3d191ec09604e43899c8b0620000338 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:02:56.529563 31141 raft_consensus.cc:729] T 1fcda57fe38545ae9f7bcc150fb6b95b P d3d191ec09604e43899c8b0620000338 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3d191ec09604e43899c8b0620000338, State: Initialized, Role: FOLLOWER
I20260811 19:02:56.529599 31139 raft_consensus.cc:729] T 1fcda57fe38545ae9f7bcc150fb6b95b P 2bd887277b2d4ff9a177548a7e23b585 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2bd887277b2d4ff9a177548a7e23b585, State: Initialized, Role: FOLLOWER
I20260811 19:02:56.530403 31141 consensus_queue.cc:260] T 1fcda57fe38545ae9f7bcc150fb6b95b P d3d191ec09604e43899c8b0620000338 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3d191ec09604e43899c8b0620000338" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 40247 } } peers { permanent_uuid: "2bd887277b2d4ff9a177548a7e23b585" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 34731 } } peers { permanent_uuid: "1ab526b793bd471390337d39a0097c47" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 35975 } }
I20260811 19:02:56.530403 31139 consensus_queue.cc:260] T 1fcda57fe38545ae9f7bcc150fb6b95b P 2bd887277b2d4ff9a177548a7e23b585 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3d191ec09604e43899c8b0620000338" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 40247 } } peers { permanent_uuid: "2bd887277b2d4ff9a177548a7e23b585" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 34731 } } peers { permanent_uuid: "1ab526b793bd471390337d39a0097c47" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 35975 } }
I20260811 19:02:56.532203 30867 heartbeater.cc:503] Master 127.29.112.190:36845 was elected leader, sending a full tablet report...
I20260811 19:02:56.533712 31141 ts_tablet_manager.cc:1428] T 1fcda57fe38545ae9f7bcc150fb6b95b P d3d191ec09604e43899c8b0620000338: Time spent starting tablet: real 0.018s user 0.011s sys 0.005s
I20260811 19:02:56.533928 31139 ts_tablet_manager.cc:1428] T 1fcda57fe38545ae9f7bcc150fb6b95b P 2bd887277b2d4ff9a177548a7e23b585: Time spent starting tablet: real 0.018s user 0.012s sys 0.005s
I20260811 19:02:56.592881 31146 raft_consensus.cc:482] T 1fcda57fe38545ae9f7bcc150fb6b95b P 1ab526b793bd471390337d39a0097c47 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:02:56.593374 31146 raft_consensus.cc:504] T 1fcda57fe38545ae9f7bcc150fb6b95b P 1ab526b793bd471390337d39a0097c47 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3d191ec09604e43899c8b0620000338" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 40247 } } peers { permanent_uuid: "2bd887277b2d4ff9a177548a7e23b585" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 34731 } } peers { permanent_uuid: "1ab526b793bd471390337d39a0097c47" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 35975 } }
I20260811 19:02:56.595677 31146 leader_election.cc:288] T 1fcda57fe38545ae9f7bcc150fb6b95b P 1ab526b793bd471390337d39a0097c47 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3d191ec09604e43899c8b0620000338 (127.29.112.130:40247), 2bd887277b2d4ff9a177548a7e23b585 (127.29.112.129:34731)
I20260811 19:02:56.605254 31112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:56.596374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.130:40247 (local address 127.29.112.131:53019)
0811 19:02:56.596722 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:56.596747 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 19:02:56.596917 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:02:56.597793 (+ 876us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:02:56.597797 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:02:56.597832 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:02:56.598073 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:56.598082 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:56.599801 (+ 1719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:56.599809 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:56.602203 (+ 2394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:56.602222 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:56.604280 (+ 2058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:56.604287 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:56.604524 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:56.604569 (+ 45us) client_negotiation.cc:769] Sending connection context
0811 19:02:56.604697 (+ 128us) client_negotiation.cc:240] Negotiation successful
0811 19:02:56.604867 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158}
I20260811 19:02:56.605394 31132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:56.596586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.131:53019 (local address 127.29.112.130:40247)
0811 19:02:56.597156 (+ 570us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:56.597165 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:56.597199 (+ 34us) server_negotiation.cc:407] Connection header received
0811 19:02:56.597340 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:56.597349 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:56.597478 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:56.597646 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:02:56.598376 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:56.599599 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:56.602442 (+ 2843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:56.604066 (+ 1624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:56.604262 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:56.604334 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:56.604785 (+ 451us) server_negotiation.cc:299] Negotiation successful
0811 19:02:56.604943 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155}
I20260811 19:02:56.606736 31138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:56.597434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.131:60701 (local address 127.29.112.129:34731)
0811 19:02:56.597684 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:56.597691 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:56.598466 (+ 775us) server_negotiation.cc:407] Connection header received
0811 19:02:56.598809 (+ 343us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:56.598814 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:56.598891 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:56.598998 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:02:56.599726 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:56.601249 (+ 1523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:56.604012 (+ 2763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:56.605431 (+ 1419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:56.605648 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:56.605748 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:56.606279 (+ 531us) server_negotiation.cc:299] Negotiation successful
0811 19:02:56.606426 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260811 19:02:56.606798 31151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:56.596983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:34731 (local address 127.29.112.131:60701)
0811 19:02:56.598176 (+ 1193us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:56.598208 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0811 19:02:56.598486 (+ 278us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:02:56.599038 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:02:56.599044 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:02:56.599072 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:02:56.599476 (+ 404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:56.599489 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:56.601481 (+ 1992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:56.601486 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:56.603652 (+ 2166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:56.603673 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:56.605852 (+ 2179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:56.605860 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:56.606074 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:56.606101 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 19:02:56.606229 (+ 128us) client_negotiation.cc:240] Negotiation successful
0811 19:02:56.606392 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":901,"thread_start_us":516,"threads_started":1}
I20260811 19:02:56.606649 30788 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fcda57fe38545ae9f7bcc150fb6b95b" candidate_uuid: "1ab526b793bd471390337d39a0097c47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3d191ec09604e43899c8b0620000338" is_pre_election: true
I20260811 19:02:56.607281 30788 raft_consensus.cc:2454] T 1fcda57fe38545ae9f7bcc150fb6b95b P d3d191ec09604e43899c8b0620000338 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ab526b793bd471390337d39a0097c47 in term 0.
I20260811 19:02:56.607867 30552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fcda57fe38545ae9f7bcc150fb6b95b" candidate_uuid: "1ab526b793bd471390337d39a0097c47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2bd887277b2d4ff9a177548a7e23b585" is_pre_election: true
I20260811 19:02:56.608103 30911 leader_election.cc:302] T 1fcda57fe38545ae9f7bcc150fb6b95b P 1ab526b793bd471390337d39a0097c47 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ab526b793bd471390337d39a0097c47, d3d191ec09604e43899c8b0620000338; no voters:
I20260811 19:02:56.608443 30552 raft_consensus.cc:2454] T 1fcda57fe38545ae9f7bcc150fb6b95b P 2bd887277b2d4ff9a177548a7e23b585 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ab526b793bd471390337d39a0097c47 in term 0.
I20260811 19:02:56.608608 31146 raft_consensus.cc:2781] T 1fcda57fe38545ae9f7bcc150fb6b95b P 1ab526b793bd471390337d39a0097c47 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 19:02:56.608770 31146 raft_consensus.cc:482] T 1fcda57fe38545ae9f7bcc150fb6b95b P 1ab526b793bd471390337d39a0097c47 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 19:02:56.608913 31146 raft_consensus.cc:3037] T 1fcda57fe38545ae9f7bcc150fb6b95b P 1ab526b793bd471390337d39a0097c47 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:02:56.611593 31146 raft_consensus.cc:504] T 1fcda57fe38545ae9f7bcc150fb6b95b P 1ab526b793bd471390337d39a0097c47 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3d191ec09604e43899c8b0620000338" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 40247 } } peers { permanent_uuid: "2bd887277b2d4ff9a177548a7e23b585" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 34731 } } peers { permanent_uuid: "1ab526b793bd471390337d39a0097c47" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 35975 } }
I20260811 19:02:56.612337 31146 leader_election.cc:288] T 1fcda57fe38545ae9f7bcc150fb6b95b P 1ab526b793bd471390337d39a0097c47 [CANDIDATE]: Term 1 election: Requested vote from peers d3d191ec09604e43899c8b0620000338 (127.29.112.130:40247), 2bd887277b2d4ff9a177548a7e23b585 (127.29.112.129:34731)
I20260811 19:02:56.612975 30788 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fcda57fe38545ae9f7bcc150fb6b95b" candidate_uuid: "1ab526b793bd471390337d39a0097c47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3d191ec09604e43899c8b0620000338"
I20260811 19:02:56.613161 30552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fcda57fe38545ae9f7bcc150fb6b95b" candidate_uuid: "1ab526b793bd471390337d39a0097c47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2bd887277b2d4ff9a177548a7e23b585"
I20260811 19:02:56.613210 30788 raft_consensus.cc:3037] T 1fcda57fe38545ae9f7bcc150fb6b95b P d3d191ec09604e43899c8b0620000338 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:02:56.613413 30552 raft_consensus.cc:3037] T 1fcda57fe38545ae9f7bcc150fb6b95b P 2bd887277b2d4ff9a177548a7e23b585 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:02:56.616057 30788 raft_consensus.cc:2454] T 1fcda57fe38545ae9f7bcc150fb6b95b P d3d191ec09604e43899c8b0620000338 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ab526b793bd471390337d39a0097c47 in term 1.
I20260811 19:02:56.616483 30552 raft_consensus.cc:2454] T 1fcda57fe38545ae9f7bcc150fb6b95b P 2bd887277b2d4ff9a177548a7e23b585 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ab526b793bd471390337d39a0097c47 in term 1.
I20260811 19:02:56.616616 30911 leader_election.cc:302] T 1fcda57fe38545ae9f7bcc150fb6b95b P 1ab526b793bd471390337d39a0097c47 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ab526b793bd471390337d39a0097c47, d3d191ec09604e43899c8b0620000338; no voters:
I20260811 19:02:56.617364 31146 raft_consensus.cc:2781] T 1fcda57fe38545ae9f7bcc150fb6b95b P 1ab526b793bd471390337d39a0097c47 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:02:56.618242 31146 raft_consensus.cc:686] T 1fcda57fe38545ae9f7bcc150fb6b95b P 1ab526b793bd471390337d39a0097c47 [term 1 LEADER]: Becoming Leader. State: Replica: 1ab526b793bd471390337d39a0097c47, State: Running, Role: LEADER
I20260811 19:02:56.619103 31146 consensus_queue.cc:237] T 1fcda57fe38545ae9f7bcc150fb6b95b P 1ab526b793bd471390337d39a0097c47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3d191ec09604e43899c8b0620000338" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 40247 } } peers { permanent_uuid: "2bd887277b2d4ff9a177548a7e23b585" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 34731 } } peers { permanent_uuid: "1ab526b793bd471390337d39a0097c47" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 35975 } }
I20260811 19:02:56.625180 30286 catalog_manager.cc:5495] T 1fcda57fe38545ae9f7bcc150fb6b95b P 1ab526b793bd471390337d39a0097c47 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1ab526b793bd471390337d39a0097c47 (127.29.112.131). New cstate: current_term: 1 leader_uuid: "1ab526b793bd471390337d39a0097c47" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3d191ec09604e43899c8b0620000338" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 40247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2bd887277b2d4ff9a177548a7e23b585" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 34731 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ab526b793bd471390337d39a0097c47" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 35975 } health_report { overall_health: HEALTHY } } }
W20260811 19:02:56.627537 31110 tablet.cc:2369] T 1fcda57fe38545ae9f7bcc150fb6b95b P 1ab526b793bd471390337d39a0097c47: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 19:02:56.670084 30870 tablet.cc:2369] T 1fcda57fe38545ae9f7bcc150fb6b95b P d3d191ec09604e43899c8b0620000338: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 19:02:56.676604 31137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:56.669035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59246 (local address 127.29.112.131:35975)
0811 19:02:56.669416 (+ 381us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:56.669422 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:56.669443 (+ 21us) server_negotiation.cc:407] Connection header received
0811 19:02:56.669539 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:56.669544 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:56.669633 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:56.669866 (+ 233us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:02:56.670446 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:56.671618 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:56.673229 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:56.673938 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:56.674077 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:56.675084 (+ 1007us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:02:56.675727 (+ 643us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:02:56.675888 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:56.676144 (+ 256us) server_negotiation.cc:299] Negotiation successful
0811 19:02:56.676242 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260811 19:02:56.687146 30552 raft_consensus.cc:1261] T 1fcda57fe38545ae9f7bcc150fb6b95b P 2bd887277b2d4ff9a177548a7e23b585 [term 1 FOLLOWER]: Refusing update from remote peer 1ab526b793bd471390337d39a0097c47: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:02:56.687438 30788 raft_consensus.cc:1261] T 1fcda57fe38545ae9f7bcc150fb6b95b P d3d191ec09604e43899c8b0620000338 [term 1 FOLLOWER]: Refusing update from remote peer 1ab526b793bd471390337d39a0097c47: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:02:56.688452 31157 consensus_queue.cc:1035] T 1fcda57fe38545ae9f7bcc150fb6b95b P 1ab526b793bd471390337d39a0097c47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2bd887277b2d4ff9a177548a7e23b585" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 34731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:02:56.689188 31146 consensus_queue.cc:1035] T 1fcda57fe38545ae9f7bcc150fb6b95b P 1ab526b793bd471390337d39a0097c47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3d191ec09604e43899c8b0620000338" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 40247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:02:56.702670 31161 mvcc.cc:205] Tried to move back new op lower bound from 7317401504495931392 to 7317401504237498368. Current Snapshot: MvccSnapshot[applied={T|T < 7317401504495931392}]
I20260811 19:02:56.707229 31166 mvcc.cc:205] Tried to move back new op lower bound from 7317401504495931392 to 7317401504237498368. Current Snapshot: MvccSnapshot[applied={T|T < 7317401504495931392}]
I20260811 19:02:56.721318 30286 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58128:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:02:56.721683 30286 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58128:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:02:56.726125 30286 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a0c12cf38d684de287a83812a1bd4d32: Sending DeleteTablet for 3 replicas of tablet 1fcda57fe38545ae9f7bcc150fb6b95b
I20260811 19:02:56.727687 30753 tablet_service.cc:1514] Processing DeleteTablet for tablet 1fcda57fe38545ae9f7bcc150fb6b95b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:02:56 UTC) from {username='kudu', principal='kudu/127.29.112.190@KRBTEST.COM'} at 127.0.0.1:47398
I20260811 19:02:56.727715 30987 tablet_service.cc:1514] Processing DeleteTablet for tablet 1fcda57fe38545ae9f7bcc150fb6b95b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:02:56 UTC) from {username='kudu', principal='kudu/127.29.112.190@KRBTEST.COM'} at 127.0.0.1:59240
I20260811 19:02:56.728838 31175 tablet_replica.cc:330] T 1fcda57fe38545ae9f7bcc150fb6b95b P d3d191ec09604e43899c8b0620000338: stopping tablet replica
I20260811 19:02:56.729048 30506 tablet_service.cc:1514] Processing DeleteTablet for tablet 1fcda57fe38545ae9f7bcc150fb6b95b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:02:56 UTC) from {username='kudu', principal='kudu/127.29.112.190@KRBTEST.COM'} at 127.0.0.1:32834
I20260811 19:02:56.729501 31175 raft_consensus.cc:2229] T 1fcda57fe38545ae9f7bcc150fb6b95b P d3d191ec09604e43899c8b0620000338 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:02:56.730206 31175 raft_consensus.cc:2258] T 1fcda57fe38545ae9f7bcc150fb6b95b P d3d191ec09604e43899c8b0620000338 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:02:56.731293 31178 tablet_replica.cc:330] T 1fcda57fe38545ae9f7bcc150fb6b95b P 2bd887277b2d4ff9a177548a7e23b585: stopping tablet replica
I20260811 19:02:56.731503 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 30402
I20260811 19:02:56.731873 31178 raft_consensus.cc:2229] T 1fcda57fe38545ae9f7bcc150fb6b95b P 2bd887277b2d4ff9a177548a7e23b585 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:02:56.732473 31178 raft_consensus.cc:2258] T 1fcda57fe38545ae9f7bcc150fb6b95b P 2bd887277b2d4ff9a177548a7e23b585 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:02:56.732722 31175 ts_tablet_manager.cc:1905] T 1fcda57fe38545ae9f7bcc150fb6b95b P d3d191ec09604e43899c8b0620000338: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:02:56.735163 31178 ts_tablet_manager.cc:1905] T 1fcda57fe38545ae9f7bcc150fb6b95b P 2bd887277b2d4ff9a177548a7e23b585: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:02:56.741834 31175 ts_tablet_manager.cc:1918] T 1fcda57fe38545ae9f7bcc150fb6b95b P d3d191ec09604e43899c8b0620000338: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:02:56.742374 31175 log.cc:1196] T 1fcda57fe38545ae9f7bcc150fb6b95b P d3d191ec09604e43899c8b0620000338: Deleting WAL directory at /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/minicluster-data/ts-1/wal/wals/1fcda57fe38545ae9f7bcc150fb6b95b
I20260811 19:02:56.743162 31175 ts_tablet_manager.cc:1939] T 1fcda57fe38545ae9f7bcc150fb6b95b P d3d191ec09604e43899c8b0620000338: Deleting consensus metadata
I20260811 19:02:56.744639 31178 ts_tablet_manager.cc:1918] T 1fcda57fe38545ae9f7bcc150fb6b95b P 2bd887277b2d4ff9a177548a7e23b585: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:02:56.744912 31178 log.cc:1196] T 1fcda57fe38545ae9f7bcc150fb6b95b P 2bd887277b2d4ff9a177548a7e23b585: Deleting WAL directory at /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/minicluster-data/ts-0/wal/wals/1fcda57fe38545ae9f7bcc150fb6b95b
I20260811 19:02:56.745528 31178 ts_tablet_manager.cc:1939] T 1fcda57fe38545ae9f7bcc150fb6b95b P 2bd887277b2d4ff9a177548a7e23b585: Deleting consensus metadata
I20260811 19:02:56.745515 30252 catalog_manager.cc:4841] TS d3d191ec09604e43899c8b0620000338 (127.29.112.130:40247): tablet 1fcda57fe38545ae9f7bcc150fb6b95b (table test-table [id=9f2b6b3bb42a4b98b1b8f722b9e47de9]) successfully deleted
I20260811 19:02:56.747290 30252 catalog_manager.cc:4841] TS 2bd887277b2d4ff9a177548a7e23b585 (127.29.112.129:34731): tablet 1fcda57fe38545ae9f7bcc150fb6b95b (table test-table [id=9f2b6b3bb42a4b98b1b8f722b9e47de9]) successfully deleted
I20260811 19:02:56.748142 31176 tablet_replica.cc:330] T 1fcda57fe38545ae9f7bcc150fb6b95b P 1ab526b793bd471390337d39a0097c47: stopping tablet replica
I20260811 19:02:56.748790 31176 raft_consensus.cc:2229] T 1fcda57fe38545ae9f7bcc150fb6b95b P 1ab526b793bd471390337d39a0097c47 [term 1 LEADER]: Raft consensus shutting down.
I20260811 19:02:56.749635 31176 raft_consensus.cc:2258] T 1fcda57fe38545ae9f7bcc150fb6b95b P 1ab526b793bd471390337d39a0097c47 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:02:56.752450 31176 ts_tablet_manager.cc:1905] T 1fcda57fe38545ae9f7bcc150fb6b95b P 1ab526b793bd471390337d39a0097c47: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 11 19:02:56 dist-test-slave-h6n0 krb5kdc[30207](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474973, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.129@KRBTEST.COM
I20260811 19:02:56.761900 31176 ts_tablet_manager.cc:1918] T 1fcda57fe38545ae9f7bcc150fb6b95b P 1ab526b793bd471390337d39a0097c47: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:02:56.762151 31176 log.cc:1196] T 1fcda57fe38545ae9f7bcc150fb6b95b P 1ab526b793bd471390337d39a0097c47: Deleting WAL directory at /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474615205622-30146-0/minicluster-data/ts-2/wal/wals/1fcda57fe38545ae9f7bcc150fb6b95b
I20260811 19:02:56.762880 31176 ts_tablet_manager.cc:1939] T 1fcda57fe38545ae9f7bcc150fb6b95b P 1ab526b793bd471390337d39a0097c47: Deleting consensus metadata
I20260811 19:02:56.765079 30258 catalog_manager.cc:4841] TS 1ab526b793bd471390337d39a0097c47 (127.29.112.131:35975): tablet 1fcda57fe38545ae9f7bcc150fb6b95b (table test-table [id=9f2b6b3bb42a4b98b1b8f722b9e47de9]) successfully deleted
I20260811 19:02:56.770372 31138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:56.735627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32838 (local address 127.29.112.129:34731)
0811 19:02:56.735902 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:56.735910 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:56.747188 (+ 11278us) server_negotiation.cc:407] Connection header received
0811 19:02:56.747454 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:56.747460 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:56.747550 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:56.747668 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:02:56.749840 (+ 2172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:56.751368 (+ 1528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:56.753548 (+ 2180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:56.754253 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:56.754402 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:56.762739 (+ 8337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:02:56.762765 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:02:56.762787 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:02:56.762867 (+ 80us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:02:56.766930 (+ 4063us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:56.768041 (+ 1111us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:56.768050 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:56.768075 (+ 25us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:56.768232 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:56.768743 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:56.768748 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:56.768750 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:56.769122 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:02:56.769356 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:56.769813 (+ 457us) server_negotiation.cc:299] Negotiation successful
0811 19:02:56.770000 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260811 19:02:56.771396 30618 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:02:56.877121 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 30402
I20260811 19:02:56.916529 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 30634
Aug 11 19:02:56 dist-test-slave-h6n0 krb5kdc[30207](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474973, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.130@KRBTEST.COM
I20260811 19:02:56.939823 31132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:56.917750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47412 (local address 127.29.112.130:40247)
0811 19:02:56.917955 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:56.917960 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:56.918103 (+ 143us) server_negotiation.cc:407] Connection header received
0811 19:02:56.918468 (+ 365us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:56.918473 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:56.918630 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:56.918730 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:02:56.921553 (+ 2823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:56.922515 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:56.925005 (+ 2490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:56.925389 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:56.925521 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:56.932131 (+ 6610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:02:56.932150 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:02:56.932167 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:02:56.932209 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:02:56.936321 (+ 4112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:56.937248 (+ 927us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:56.937260 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:56.937263 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:56.937399 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:56.937856 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:56.937863 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:56.937866 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:56.938306 (+ 440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:02:56.938696 (+ 390us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:56.939169 (+ 473us) server_negotiation.cc:299] Negotiation successful
0811 19:02:56.939421 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 19:02:56.941047 30855 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:02:57.042970 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 30634
I20260811 19:02:57.069600 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 30879
Aug 11 19:02:57 dist-test-slave-h6n0 krb5kdc[30207](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474973, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.131@KRBTEST.COM
I20260811 19:02:57.089754 31137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:57.070701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59252 (local address 127.29.112.131:35975)
0811 19:02:57.070913 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:57.070921 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:57.070949 (+ 28us) server_negotiation.cc:407] Connection header received
0811 19:02:57.071182 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:57.071195 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:57.071283 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:57.071403 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:02:57.072871 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:57.074189 (+ 1318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:57.075914 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:57.076593 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:57.076757 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:57.081755 (+ 4998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:02:57.081768 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:02:57.081784 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:02:57.081840 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:02:57.086118 (+ 4278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:57.087074 (+ 956us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:57.087082 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:57.087085 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:57.087262 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:02:57.087718 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:02:57.087721 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:02:57.087724 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:02:57.088238 (+ 514us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:02:57.088617 (+ 379us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:57.089017 (+ 400us) server_negotiation.cc:299] Negotiation successful
0811 19:02:57.089279 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260811 19:02:57.090651 31097 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:02:57.232427 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 30879
I20260811 19:02:57.265725 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 30229
I20260811 19:02:57.267160 30345 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:02:57.391004 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 30229
2026-08-11T19:02:57Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (5918 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-11T19:02:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:02:57Z Disabled control of system clock
I20260811 19:02:57.505333 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474615205622-30146-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474615205622-30146-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474615205622-30146-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.190:34291
--webserver_interface=127.29.112.190
--webserver_port=0
--builtin_ntp_servers=127.29.112.148:38945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.190:34291
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 19:02:57.888396 31199 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:02:57.888742 31199 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:02:57.888885 31199 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:02:57.897894 31199 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:02:57.898033 31199 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:02:57.898130 31199 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:02:57.898253 31199 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:02:57.898350 31199 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:02:57.909837 31199 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:38945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474615205622-30146-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474615205622-30146-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.112.190:34291
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.190:34291
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.112.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474615205622-30146-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:02:57.911384 31199 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:02:57.913471 31199 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:02:57.926945 31211 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:02:57.926932 31217 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:02:57.927052 31209 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:02:57.928174 31199 server_base.cc:1034] running on GCE node
I20260811 19:02:57.929184 31199 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:02:57.930974 31199 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:02:57.932273 31199 hybrid_clock.cc:648] HybridClock initialized: now 1786474977932221 us; error 49 us; skew 500 ppm
I20260811 19:02:57.932726 31199 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 19:02:57.935756 31199 webserver.cc:460] Webserver started at http://127.29.112.190:37071/ using document root <none> and password file <none>
I20260811 19:02:57.936906 31199 fs_manager.cc:359] Metadata directory not provided
I20260811 19:02:57.937141 31199 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:02:57.937600 31199 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:02:57.941448 31199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474615205622-30146-0/minicluster-data/master-0/data/instance:
uuid: "42b24f555a064b248a9bd4a01c808016"
format_stamp: "Formatted at 2026-08-11 19:02:57 on dist-test-slave-h6n0"
server_key: "67c147f8cf81b89226a22d2b54a44494"
server_key_iv: "366602db63f67008a80e7466b53830e4"
server_key_version: "encryptionkey@0"
I20260811 19:02:57.942452 31199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474615205622-30146-0/minicluster-data/master-0/wal/instance:
uuid: "42b24f555a064b248a9bd4a01c808016"
format_stamp: "Formatted at 2026-08-11 19:02:57 on dist-test-slave-h6n0"
server_key: "67c147f8cf81b89226a22d2b54a44494"
server_key_iv: "366602db63f67008a80e7466b53830e4"
server_key_version: "encryptionkey@0"
I20260811 19:02:57.950675 31199 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.006s sys 0.001s
I20260811 19:02:57.955670 31225 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:57.957470 31199 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260811 19:02:57.957899 31199 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474615205622-30146-0/minicluster-data/master-0/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474615205622-30146-0/minicluster-data/master-0/wal
uuid: "42b24f555a064b248a9bd4a01c808016"
format_stamp: "Formatted at 2026-08-11 19:02:57 on dist-test-slave-h6n0"
server_key: "67c147f8cf81b89226a22d2b54a44494"
server_key_iv: "366602db63f67008a80e7466b53830e4"
server_key_version: "encryptionkey@0"
I20260811 19:02:57.958256 31199 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474615205622-30146-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474615205622-30146-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474615205622-30146-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:02:58.007759 31199 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:02:58.008744 31199 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:02:58.009055 31199 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:02:58.029489 31199 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.190:34291
I20260811 19:02:58.029536 31325 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.190:34291 every 8 connection(s)
I20260811 19:02:58.030893 31199 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
I20260811 19:02:58.034946 31326 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:02:58.035521 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 31199
I20260811 19:02:58.035730 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474615205622-30146-0/minicluster-data/master-0/wal/instance
I20260811 19:02:58.036405 30146 external_mini_cluster.cc:1428] Setting key 4deb6dd2e5ab92b80c8807017e8e6ebe
I20260811 19:02:58.050037 31326 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42b24f555a064b248a9bd4a01c808016: Bootstrap starting.
I20260811 19:02:58.052954 31330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:58.038921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42836 (local address 127.29.112.190:34291)
0811 19:02:58.040076 (+ 1155us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:58.040087 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:58.040134 (+ 47us) server_negotiation.cc:407] Connection header received
0811 19:02:58.042266 (+ 2132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:58.042298 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:58.042912 (+ 614us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:58.043255 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:02:58.043878 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:58.045206 (+ 1328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:58.047215 (+ 2009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:58.048079 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:58.048237 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:58.049900 (+ 1663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:02:58.049908 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:02:58.049922 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:02:58.050642 (+ 720us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:02:58.050757 (+ 115us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:02:58.050781 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:02:58.050977 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:58.051487 (+ 510us) server_negotiation.cc:299] Negotiation successful
0811 19:02:58.051976 (+ 489us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":713,"thread_start_us":453,"threads_started":1}
I20260811 19:02:58.053876 31326 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 42b24f555a064b248a9bd4a01c808016: Neither blocks nor log segments found. Creating new log.
I20260811 19:02:58.055276 31326 log.cc:824] T 00000000000000000000000000000000 P 42b24f555a064b248a9bd4a01c808016: Log is configured to *not* fsync() on all Append() calls
I20260811 19:02:58.058748 31326 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42b24f555a064b248a9bd4a01c808016: No bootstrap required, opened a new log
I20260811 19:02:58.067960 31326 raft_consensus.cc:348] T 00000000000000000000000000000000 P 42b24f555a064b248a9bd4a01c808016 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42b24f555a064b248a9bd4a01c808016" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 34291 } }
I20260811 19:02:58.068804 31326 raft_consensus.cc:374] T 00000000000000000000000000000000 P 42b24f555a064b248a9bd4a01c808016 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:02:58.068950 31326 raft_consensus.cc:729] T 00000000000000000000000000000000 P 42b24f555a064b248a9bd4a01c808016 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42b24f555a064b248a9bd4a01c808016, State: Initialized, Role: FOLLOWER
I20260811 19:02:58.070212 31326 consensus_queue.cc:260] T 00000000000000000000000000000000 P 42b24f555a064b248a9bd4a01c808016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42b24f555a064b248a9bd4a01c808016" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 34291 } }
I20260811 19:02:58.070708 31326 raft_consensus.cc:388] T 00000000000000000000000000000000 P 42b24f555a064b248a9bd4a01c808016 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:02:58.070879 31326 raft_consensus.cc:482] T 00000000000000000000000000000000 P 42b24f555a064b248a9bd4a01c808016 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:02:58.071076 31326 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 42b24f555a064b248a9bd4a01c808016 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:02:58.073462 31326 raft_consensus.cc:504] T 00000000000000000000000000000000 P 42b24f555a064b248a9bd4a01c808016 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42b24f555a064b248a9bd4a01c808016" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 34291 } }
I20260811 19:02:58.074075 31326 leader_election.cc:302] T 00000000000000000000000000000000 P 42b24f555a064b248a9bd4a01c808016 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42b24f555a064b248a9bd4a01c808016; no voters:
I20260811 19:02:58.074772 31326 leader_election.cc:288] T 00000000000000000000000000000000 P 42b24f555a064b248a9bd4a01c808016 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:02:58.074904 31332 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 42b24f555a064b248a9bd4a01c808016 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:02:58.076597 31332 raft_consensus.cc:686] T 00000000000000000000000000000000 P 42b24f555a064b248a9bd4a01c808016 [term 1 LEADER]: Becoming Leader. State: Replica: 42b24f555a064b248a9bd4a01c808016, State: Running, Role: LEADER
I20260811 19:02:58.077358 31326 sys_catalog.cc:564] T 00000000000000000000000000000000 P 42b24f555a064b248a9bd4a01c808016 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:02:58.077566 31332 consensus_queue.cc:237] T 00000000000000000000000000000000 P 42b24f555a064b248a9bd4a01c808016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42b24f555a064b248a9bd4a01c808016" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 34291 } }
I20260811 19:02:58.083133 31333 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42b24f555a064b248a9bd4a01c808016 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "42b24f555a064b248a9bd4a01c808016" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42b24f555a064b248a9bd4a01c808016" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 34291 } } }
I20260811 19:02:58.083740 31333 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42b24f555a064b248a9bd4a01c808016 [sys.catalog]: This master's current role is: LEADER
I20260811 19:02:58.084578 31334 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42b24f555a064b248a9bd4a01c808016 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 42b24f555a064b248a9bd4a01c808016. Latest consensus state: current_term: 1 leader_uuid: "42b24f555a064b248a9bd4a01c808016" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42b24f555a064b248a9bd4a01c808016" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 34291 } } }
I20260811 19:02:58.085107 31334 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42b24f555a064b248a9bd4a01c808016 [sys.catalog]: This master's current role is: LEADER
I20260811 19:02:58.087795 31349 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:02:58.093235 31349 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:02:58.103821 31349 catalog_manager.cc:1347] Generated new cluster ID: 095065f8e4fa4efca94abad77351c667
I20260811 19:02:58.103965 31349 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:02:58.127261 31349 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:02:58.129176 31349 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:02:58.144068 31349 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 42b24f555a064b248a9bd4a01c808016: Generated new TSK 0
I20260811 19:02:58.145107 31349 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 19:02:58.160243 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474615205622-30146-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474615205622-30146-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474615205622-30146-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474615205622-30146-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.129:0
--local_ip_for_outbound_sockets=127.29.112.129
--webserver_interface=127.29.112.129
--webserver_port=0
--tserver_master_addrs=127.29.112.190:34291
--builtin_ntp_servers=127.29.112.148:38945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 19:02:58.621284 31365 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:02:58.621588 31365 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:02:58.621681 31365 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:02:58.630945 31365 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:02:58.631067 31365 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:02:58.631177 31365 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.129
I20260811 19:02:58.640578 31365 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:38945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474615205622-30146-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474615205622-30146-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474615205622-30146-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.112.129
--webserver_port=0
--tserver_master_addrs=127.29.112.190:34291
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.129
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474615205622-30146-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:02:58.641987 31365 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:02:58.644208 31365 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:02:58.659580 31372 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:02:58.659719 31377 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:02:58.660297 31373 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:02:58.660561 31365 server_base.cc:1034] running on GCE node
I20260811 19:02:58.661293 31365 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:02:58.662343 31365 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:02:58.663600 31365 hybrid_clock.cc:648] HybridClock initialized: now 1786474978663543 us; error 66 us; skew 500 ppm
I20260811 19:02:58.663969 31365 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 19:02:58.667100 31365 webserver.cc:460] Webserver started at http://127.29.112.129:36999/ using document root <none> and password file <none>
I20260811 19:02:58.667819 31365 fs_manager.cc:359] Metadata directory not provided
I20260811 19:02:58.667955 31365 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:02:58.668305 31365 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:02:58.671226 31365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474615205622-30146-0/minicluster-data/ts-0/data/instance:
uuid: "9f87c124504d4d4699185ef2652e201f"
format_stamp: "Formatted at 2026-08-11 19:02:58 on dist-test-slave-h6n0"
server_key: "922d56a468d6b96c7fc0c12f8f829779"
server_key_iv: "3e62ebc869217653f9c2a51d2b94cb8b"
server_key_version: "encryptionkey@0"
I20260811 19:02:58.672032 31365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474615205622-30146-0/minicluster-data/ts-0/wal/instance:
uuid: "9f87c124504d4d4699185ef2652e201f"
format_stamp: "Formatted at 2026-08-11 19:02:58 on dist-test-slave-h6n0"
server_key: "922d56a468d6b96c7fc0c12f8f829779"
server_key_iv: "3e62ebc869217653f9c2a51d2b94cb8b"
server_key_version: "encryptionkey@0"
I20260811 19:02:58.677526 31365 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 19:02:58.681643 31391 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:58.683662 31365 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 19:02:58.683857 31365 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474615205622-30146-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474615205622-30146-0/minicluster-data/ts-0/wal
uuid: "9f87c124504d4d4699185ef2652e201f"
format_stamp: "Formatted at 2026-08-11 19:02:58 on dist-test-slave-h6n0"
server_key: "922d56a468d6b96c7fc0c12f8f829779"
server_key_iv: "3e62ebc869217653f9c2a51d2b94cb8b"
server_key_version: "encryptionkey@0"
I20260811 19:02:58.684114 31365 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474615205622-30146-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474615205622-30146-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474615205622-30146-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:02:58.704937 31365 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:02:58.705806 31365 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:02:58.706144 31365 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:02:58.707239 31365 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:02:58.708777 31365 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:02:58.708906 31365 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:58.709046 31365 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:02:58.709132 31365 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:58.746156 31365 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.129:38943
I20260811 19:02:58.746620 31584 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.129:38943 every 8 connection(s)
I20260811 19:02:58.747750 31365 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474615205622-30146-0/minicluster-data/ts-0/data/info.pb
I20260811 19:02:58.758147 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 31365
I20260811 19:02:58.758365 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474615205622-30146-0/minicluster-data/ts-0/wal/instance
I20260811 19:02:58.759021 30146 external_mini_cluster.cc:1428] Setting key b8077c8e42fc934655eaeb05a5a8bd53
I20260811 19:02:58.762290 31587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:58.750710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.129:33103 (local address 127.29.112.190:34291)
0811 19:02:58.751623 (+ 913us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:58.751628 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:58.752905 (+ 1277us) server_negotiation.cc:407] Connection header received
0811 19:02:58.754927 (+ 2022us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:58.754933 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:58.755025 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:58.755154 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:02:58.756244 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:58.757060 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:58.758903 (+ 1843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:58.759609 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:58.759753 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:58.761351 (+ 1598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:02:58.761359 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:02:58.761364 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:02:58.761385 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:02:58.761440 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:02:58.761458 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:02:58.761578 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:58.761793 (+ 215us) server_negotiation.cc:299] Negotiation successful
0811 19:02:58.761894 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":790,"thread_start_us":653,"threads_started":1}
I20260811 19:02:58.762667 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474615205622-30146-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474615205622-30146-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474615205622-30146-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474615205622-30146-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.130:0
--local_ip_for_outbound_sockets=127.29.112.130
--webserver_interface=127.29.112.130
--webserver_port=0
--tserver_master_addrs=127.29.112.190:34291
--builtin_ntp_servers=127.29.112.148:38945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 19:02:58.763295 31588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:58.751232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:34291 (local address 127.29.112.129:33103)
0811 19:02:58.752667 (+ 1435us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:58.752742 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0811 19:02:58.754645 (+ 1903us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:02:58.755362 (+ 717us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:02:58.755370 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:02:58.755491 (+ 121us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:02:58.756079 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:58.756095 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:58.757233 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:58.757237 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:58.758734 (+ 1497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:58.758746 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:58.759873 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:58.759883 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:58.760141 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:58.760963 (+ 822us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 19:02:58.760968 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:02:58.761025 (+ 57us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 19:02:58.761134 (+ 109us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:02:58.761575 (+ 441us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:02:58.761586 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:02:58.761594 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 19:02:58.761737 (+ 143us) client_negotiation.cc:240] Negotiation successful
0811 19:02:58.762017 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":675,"thread_start_us":442,"threads_started":1}
I20260811 19:02:58.765049 31585 heartbeater.cc:347] Connected to a master server at 127.29.112.190:34291
I20260811 19:02:58.765573 31585 heartbeater.cc:464] Registering TS with master...
I20260811 19:02:58.766638 31585 heartbeater.cc:511] Master 127.29.112.190:34291 requested a full tablet report, sending...
I20260811 19:02:58.768851 31262 ts_manager.cc:194] Registered new tserver with Master: 9f87c124504d4d4699185ef2652e201f (127.29.112.129:38943)
I20260811 19:02:58.770431 31262 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.112.129:33103
W20260811 19:02:59.160943 31591 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:02:59.161295 31591 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:02:59.161463 31591 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:02:59.169009 31591 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:02:59.169128 31591 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:02:59.169313 31591 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.130
I20260811 19:02:59.179633 31591 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:38945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474615205622-30146-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474615205622-30146-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474615205622-30146-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.112.130
--webserver_port=0
--tserver_master_addrs=127.29.112.190:34291
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.130
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474615205622-30146-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:02:59.180974 31591 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:02:59.182850 31591 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:02:59.197850 31601 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:02:59.198112 31602 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:02:59.198382 31607 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:02:59.199409 31591 server_base.cc:1034] running on GCE node
I20260811 19:02:59.200183 31591 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:02:59.201889 31591 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:02:59.203190 31591 hybrid_clock.cc:648] HybridClock initialized: now 1786474979203088 us; error 97 us; skew 500 ppm
I20260811 19:02:59.203595 31591 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 19:02:59.211153 31591 webserver.cc:460] Webserver started at http://127.29.112.130:43937/ using document root <none> and password file <none>
I20260811 19:02:59.211961 31591 fs_manager.cc:359] Metadata directory not provided
I20260811 19:02:59.212060 31591 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:02:59.212385 31591 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:02:59.215515 31591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474615205622-30146-0/minicluster-data/ts-1/data/instance:
uuid: "5dcf1b86861b46ea8173b48801406752"
format_stamp: "Formatted at 2026-08-11 19:02:59 on dist-test-slave-h6n0"
server_key: "539f9728d523f31bdca3a61533ab8d4a"
server_key_iv: "8bab86f3aeb2367504fdf922683c1b55"
server_key_version: "encryptionkey@0"
I20260811 19:02:59.216356 31591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474615205622-30146-0/minicluster-data/ts-1/wal/instance:
uuid: "5dcf1b86861b46ea8173b48801406752"
format_stamp: "Formatted at 2026-08-11 19:02:59 on dist-test-slave-h6n0"
server_key: "539f9728d523f31bdca3a61533ab8d4a"
server_key_iv: "8bab86f3aeb2367504fdf922683c1b55"
server_key_version: "encryptionkey@0"
I20260811 19:02:59.222204 31591 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 19:02:59.225979 31621 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:59.227847 31591 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 19:02:59.228089 31591 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474615205622-30146-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474615205622-30146-0/minicluster-data/ts-1/wal
uuid: "5dcf1b86861b46ea8173b48801406752"
format_stamp: "Formatted at 2026-08-11 19:02:59 on dist-test-slave-h6n0"
server_key: "539f9728d523f31bdca3a61533ab8d4a"
server_key_iv: "8bab86f3aeb2367504fdf922683c1b55"
server_key_version: "encryptionkey@0"
I20260811 19:02:59.228333 31591 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474615205622-30146-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474615205622-30146-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474615205622-30146-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:02:59.252175 31591 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:02:59.253142 31591 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:02:59.253497 31591 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:02:59.254733 31591 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:02:59.256981 31591 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:02:59.257133 31591 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:59.257277 31591 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:02:59.257355 31591 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:59.296702 31591 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.130:33947
I20260811 19:02:59.296746 31831 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.130:33947 every 8 connection(s)
I20260811 19:02:59.298061 31591 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474615205622-30146-0/minicluster-data/ts-1/data/info.pb
I20260811 19:02:59.306038 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 31591
I20260811 19:02:59.306247 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474615205622-30146-0/minicluster-data/ts-1/wal/instance
I20260811 19:02:59.306851 30146 external_mini_cluster.cc:1428] Setting key 79b5bd02ff09d931f6898c3f1981a760
I20260811 19:02:59.310022 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474615205622-30146-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474615205622-30146-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474615205622-30146-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474615205622-30146-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.112.131:0
--local_ip_for_outbound_sockets=127.29.112.131
--webserver_interface=127.29.112.131
--webserver_port=0
--tserver_master_addrs=127.29.112.190:34291
--builtin_ntp_servers=127.29.112.148:38945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 19:02:59.312640 31843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:59.300739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.130:35869 (local address 127.29.112.190:34291)
0811 19:02:59.301224 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:59.301228 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:59.302250 (+ 1022us) server_negotiation.cc:407] Connection header received
0811 19:02:59.304159 (+ 1909us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:59.304165 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:59.304263 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:59.304379 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:02:59.305404 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:59.306481 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:59.308834 (+ 2353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:59.309424 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:59.309560 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:59.311489 (+ 1929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:02:59.311498 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:02:59.311506 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:02:59.311532 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:02:59.311594 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:02:59.311614 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:02:59.311725 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:59.312069 (+ 344us) server_negotiation.cc:299] Negotiation successful
0811 19:02:59.312187 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":277,"threads_started":1}
I20260811 19:02:59.313581 31844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:59.301177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:34291 (local address 127.29.112.130:35869)
0811 19:02:59.302084 (+ 907us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:59.302128 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 19:02:59.303895 (+ 1767us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:02:59.304603 (+ 708us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:02:59.304610 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:02:59.304690 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:02:59.305237 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:59.305254 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:59.306812 (+ 1558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:59.306816 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:59.308645 (+ 1829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:59.308668 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:59.309644 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:59.309649 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:59.309860 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:59.310922 (+ 1062us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 19:02:59.310926 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:02:59.310998 (+ 72us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 19:02:59.311195 (+ 197us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:02:59.311791 (+ 596us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:02:59.311802 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:02:59.311810 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 19:02:59.312001 (+ 191us) client_negotiation.cc:240] Negotiation successful
0811 19:02:59.312352 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":303,"threads_started":1}
I20260811 19:02:59.315191 31833 heartbeater.cc:347] Connected to a master server at 127.29.112.190:34291
I20260811 19:02:59.315531 31833 heartbeater.cc:464] Registering TS with master...
I20260811 19:02:59.316515 31833 heartbeater.cc:511] Master 127.29.112.190:34291 requested a full tablet report, sending...
I20260811 19:02:59.318220 31262 ts_manager.cc:194] Registered new tserver with Master: 5dcf1b86861b46ea8173b48801406752 (127.29.112.130:33947)
I20260811 19:02:59.319370 31262 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.112.130:35869
W20260811 19:02:59.685179 31845 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:02:59.685488 31845 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:02:59.685652 31845 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:02:59.693596 31845 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:02:59.693710 31845 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:02:59.693892 31845 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.112.131
I20260811 19:02:59.704259 31845 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:38945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474615205622-30146-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474615205622-30146-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.112.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474615205622-30146-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.112.131
--webserver_port=0
--tserver_master_addrs=127.29.112.190:34291
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.112.131
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474615205622-30146-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:02:59.705756 31845 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:02:59.707683 31845 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:02:59.723089 31859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:02:59.723066 31853 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:02:59.723564 31852 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:02:59.728822 31845 server_base.cc:1034] running on GCE node
I20260811 19:02:59.729934 31845 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:02:59.731693 31845 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:02:59.732967 31845 hybrid_clock.cc:648] HybridClock initialized: now 1786474979732888 us; error 70 us; skew 500 ppm
I20260811 19:02:59.733398 31845 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 19:02:59.736289 31845 webserver.cc:460] Webserver started at http://127.29.112.131:36917/ using document root <none> and password file <none>
I20260811 19:02:59.737123 31845 fs_manager.cc:359] Metadata directory not provided
I20260811 19:02:59.737457 31845 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:02:59.737849 31845 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:02:59.741029 31845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474615205622-30146-0/minicluster-data/ts-2/data/instance:
uuid: "3be6873b0693445f8d35457b87bab346"
format_stamp: "Formatted at 2026-08-11 19:02:59 on dist-test-slave-h6n0"
server_key: "86e16b87b943304fc6d4a3d9933248aa"
server_key_iv: "5c0dcfc1b30d7fc396b5eb46fa9a3a29"
server_key_version: "encryptionkey@0"
I20260811 19:02:59.741874 31845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474615205622-30146-0/minicluster-data/ts-2/wal/instance:
uuid: "3be6873b0693445f8d35457b87bab346"
format_stamp: "Formatted at 2026-08-11 19:02:59 on dist-test-slave-h6n0"
server_key: "86e16b87b943304fc6d4a3d9933248aa"
server_key_iv: "5c0dcfc1b30d7fc396b5eb46fa9a3a29"
server_key_version: "encryptionkey@0"
I20260811 19:02:59.748373 31845 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260811 19:02:59.752563 31874 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:59.754058 31845 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 19:02:59.754356 31845 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474615205622-30146-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474615205622-30146-0/minicluster-data/ts-2/wal
uuid: "3be6873b0693445f8d35457b87bab346"
format_stamp: "Formatted at 2026-08-11 19:02:59 on dist-test-slave-h6n0"
server_key: "86e16b87b943304fc6d4a3d9933248aa"
server_key_iv: "5c0dcfc1b30d7fc396b5eb46fa9a3a29"
server_key_version: "encryptionkey@0"
I20260811 19:02:59.754715 31845 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474615205622-30146-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474615205622-30146-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474615205622-30146-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:02:59.770663 31845 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:02:59.772076 31845 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:02:59.772660 31845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:02:59.773486 31585 heartbeater.cc:503] Master 127.29.112.190:34291 was elected leader, sending a full tablet report...
I20260811 19:02:59.774004 31845 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:02:59.776520 31845 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:02:59.776671 31845 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:59.776858 31845 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:02:59.776970 31845 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:02:59.816819 31845 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.131:39793
I20260811 19:02:59.816875 32065 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.131:39793 every 8 connection(s)
I20260811 19:02:59.818231 31845 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474615205622-30146-0/minicluster-data/ts-2/data/info.pb
I20260811 19:02:59.821617 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 31845
I20260811 19:02:59.821803 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474615205622-30146-0/minicluster-data/ts-2/wal/instance
I20260811 19:02:59.822399 30146 external_mini_cluster.cc:1428] Setting key accb41ad93691a65ecfe89f3b9186280
I20260811 19:02:59.832922 32069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:59.820993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.131:55839 (local address 127.29.112.190:34291)
0811 19:02:59.821485 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:59.821493 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:59.822960 (+ 1467us) server_negotiation.cc:407] Connection header received
0811 19:02:59.825185 (+ 2225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:59.825192 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:59.825289 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:59.825422 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:02:59.826657 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:59.827498 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:59.829296 (+ 1798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:59.829723 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:59.829827 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:59.831611 (+ 1784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:02:59.831618 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:02:59.831622 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:02:59.831650 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:02:59.831720 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:02:59.831737 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:02:59.831865 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:59.832253 (+ 388us) server_negotiation.cc:299] Negotiation successful
0811 19:02:59.832387 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":228,"threads_started":1}
I20260811 19:02:59.833588 32070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:59.821547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:34291 (local address 127.29.112.131:55839)
0811 19:02:59.822755 (+ 1208us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:59.822816 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 19:02:59.824908 (+ 2092us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:02:59.825716 (+ 808us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:02:59.825725 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:02:59.825836 (+ 111us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:02:59.826441 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:59.826457 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:59.827677 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:59.827682 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:59.829107 (+ 1425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:59.829121 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:59.829943 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:59.829948 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:59.830155 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:59.830948 (+ 793us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 19:02:59.830956 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:02:59.831003 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 19:02:59.831091 (+ 88us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:02:59.831933 (+ 842us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:02:59.831942 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:02:59.831949 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 19:02:59.832112 (+ 163us) client_negotiation.cc:240] Negotiation successful
0811 19:02:59.832411 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":595,"thread_start_us":388,"threads_started":1}
I20260811 19:02:59.835201 32067 heartbeater.cc:347] Connected to a master server at 127.29.112.190:34291
I20260811 19:02:59.835551 32067 heartbeater.cc:464] Registering TS with master...
I20260811 19:02:59.836410 32067 heartbeater.cc:511] Master 127.29.112.190:34291 requested a full tablet report, sending...
I20260811 19:02:59.838439 31261 ts_manager.cc:194] Registered new tserver with Master: 3be6873b0693445f8d35457b87bab346 (127.29.112.131:39793)
I20260811 19:02:59.839442 31261 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.112.131:55839
I20260811 19:02:59.851579 30146 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 19:02:59.862363 32069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:59.856236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49650 (local address 127.29.112.190:34291)
0811 19:02:59.856421 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:59.856426 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:59.856948 (+ 522us) server_negotiation.cc:407] Connection header received
0811 19:02:59.857097 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:59.857101 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:59.857169 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:59.857259 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:02:59.857779 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:59.858502 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:59.860015 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:59.860405 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:59.860544 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:59.861616 (+ 1072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:02:59.861621 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:02:59.861625 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:02:59.861642 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:02:59.861676 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:02:59.861689 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:02:59.861789 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:59.861943 (+ 154us) server_negotiation.cc:299] Negotiation successful
0811 19:02:59.862041 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 19:02:59.867075 31261 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49650:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 19:02:59.874130 31261 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 19:02:59.899274 32092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:59.888654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34010 (local address 127.29.112.131:39793)
0811 19:02:59.890844 (+ 2190us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:59.890857 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:59.890903 (+ 46us) server_negotiation.cc:407] Connection header received
0811 19:02:59.891158 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:59.891172 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:59.891519 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:59.891789 (+ 270us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:02:59.892598 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:59.894014 (+ 1416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:59.896340 (+ 2326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:59.897752 (+ 1412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:59.898003 (+ 251us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:59.898262 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:59.898691 (+ 429us) server_negotiation.cc:299] Negotiation successful
0811 19:02:59.898873 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1966,"thread_start_us":1762,"threads_started":1}
I20260811 19:02:59.899432 32090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:59.888099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.131:39793 (local address 127.0.0.1:34010)
0811 19:02:59.888992 (+ 893us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:59.889051 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 19:02:59.889249 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:02:59.891829 (+ 2580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:02:59.891842 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:02:59.891911 (+ 69us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:02:59.892396 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:59.892414 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:59.894159 (+ 1745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:59.894165 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:59.895825 (+ 1660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:59.895839 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:59.898011 (+ 2172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:59.898023 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:59.898311 (+ 288us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:59.898443 (+ 132us) client_negotiation.cc:769] Sending connection context
0811 19:02:59.898645 (+ 202us) client_negotiation.cc:240] Negotiation successful
0811 19:02:59.898929 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"mutex_wait_us":36,"spinlock_wait_cycles":6784,"thread_start_us":842,"threads_started":1}
I20260811 19:02:59.903123 32093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:59.889118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.130:33947 (local address 127.0.0.1:44270)
0811 19:02:59.889823 (+ 705us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:59.889842 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 19:02:59.889972 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:02:59.893468 (+ 3496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:02:59.893477 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:02:59.893519 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:02:59.894081 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:59.894099 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:59.897833 (+ 3734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:59.897840 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:59.900112 (+ 2272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:59.900126 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:59.902234 (+ 2108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:59.902243 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:59.902458 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:59.902494 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 19:02:59.902619 (+ 125us) client_negotiation.cc:240] Negotiation successful
0811 19:02:59.902763 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":589,"mutex_wait_us":1,"thread_start_us":790,"threads_started":1}
I20260811 19:02:59.903270 32094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:59.889678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44270 (local address 127.29.112.130:33947)
0811 19:02:59.892693 (+ 3015us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:59.892706 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:59.892742 (+ 36us) server_negotiation.cc:407] Connection header received
0811 19:02:59.892879 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:59.892899 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:59.893179 (+ 280us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:59.894348 (+ 1169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:02:59.894666 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:59.896902 (+ 2236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:59.900316 (+ 3414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:59.901995 (+ 1679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:59.902177 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:59.902435 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:59.902685 (+ 250us) server_negotiation.cc:299] Negotiation successful
0811 19:02:59.902876 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2782,"thread_start_us":2624,"threads_started":1}
I20260811 19:02:59.904341 31956 tablet_service.cc:1467] Processing CreateTablet for tablet 3543092451244c62b56ac491d3c3afbe (DEFAULT_TABLE table=test-table [id=b044650450514c1eaa5536dc8ac8f847]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:02:59.905982 31956 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3543092451244c62b56ac491d3c3afbe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:02:59.907330 32091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:59.888342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:38943 (local address 127.0.0.1:41566)
0811 19:02:59.889051 (+ 709us) negotiation.cc:107] Waiting for socket to connect
0811 19:02:59.889066 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 19:02:59.889249 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:02:59.898443 (+ 9194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:02:59.898452 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:02:59.898481 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:02:59.899096 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:59.899112 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:59.901131 (+ 2019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:59.901139 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:59.903877 (+ 2738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:02:59.903910 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:02:59.906468 (+ 2558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:02:59.906477 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:02:59.906701 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:59.906733 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 19:02:59.906825 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 19:02:59.906987 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":580,"spinlock_wait_cycles":2176,"thread_start_us":1732,"threads_started":1}
I20260811 19:02:59.908087 32098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:02:59.890123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41566 (local address 127.29.112.129:38943)
0811 19:02:59.897529 (+ 7406us) server_negotiation.cc:206] Beginning negotiation
0811 19:02:59.897535 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:02:59.897560 (+ 25us) server_negotiation.cc:407] Connection header received
0811 19:02:59.897698 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:02:59.897709 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:02:59.897950 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:02:59.898159 (+ 209us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:02:59.899336 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:59.900940 (+ 1604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:59.904368 (+ 3428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:02:59.905901 (+ 1533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:02:59.906115 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:02:59.906316 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0811 19:02:59.907621 (+ 1305us) server_negotiation.cc:299] Negotiation successful
0811 19:02:59.907793 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7222,"thread_start_us":7073,"threads_started":1}
I20260811 19:02:59.908142 31724 tablet_service.cc:1467] Processing CreateTablet for tablet 3543092451244c62b56ac491d3c3afbe (DEFAULT_TABLE table=test-table [id=b044650450514c1eaa5536dc8ac8f847]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:02:59.909466 31724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3543092451244c62b56ac491d3c3afbe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:02:59.910938 31477 tablet_service.cc:1467] Processing CreateTablet for tablet 3543092451244c62b56ac491d3c3afbe (DEFAULT_TABLE table=test-table [id=b044650450514c1eaa5536dc8ac8f847]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:02:59.912058 31477 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3543092451244c62b56ac491d3c3afbe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:02:59.921813 32099 tablet_bootstrap.cc:492] T 3543092451244c62b56ac491d3c3afbe P 3be6873b0693445f8d35457b87bab346: Bootstrap starting.
I20260811 19:02:59.926136 32099 tablet_bootstrap.cc:654] T 3543092451244c62b56ac491d3c3afbe P 3be6873b0693445f8d35457b87bab346: Neither blocks nor log segments found. Creating new log.
I20260811 19:02:59.927497 32101 tablet_bootstrap.cc:492] T 3543092451244c62b56ac491d3c3afbe P 9f87c124504d4d4699185ef2652e201f: Bootstrap starting.
I20260811 19:02:59.927595 32099 log.cc:824] T 3543092451244c62b56ac491d3c3afbe P 3be6873b0693445f8d35457b87bab346: Log is configured to *not* fsync() on all Append() calls
I20260811 19:02:59.928416 32100 tablet_bootstrap.cc:492] T 3543092451244c62b56ac491d3c3afbe P 5dcf1b86861b46ea8173b48801406752: Bootstrap starting.
I20260811 19:02:59.931911 32100 tablet_bootstrap.cc:654] T 3543092451244c62b56ac491d3c3afbe P 5dcf1b86861b46ea8173b48801406752: Neither blocks nor log segments found. Creating new log.
I20260811 19:02:59.932282 32099 tablet_bootstrap.cc:492] T 3543092451244c62b56ac491d3c3afbe P 3be6873b0693445f8d35457b87bab346: No bootstrap required, opened a new log
I20260811 19:02:59.932590 32099 ts_tablet_manager.cc:1397] T 3543092451244c62b56ac491d3c3afbe P 3be6873b0693445f8d35457b87bab346: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260811 19:02:59.932724 32101 tablet_bootstrap.cc:654] T 3543092451244c62b56ac491d3c3afbe P 9f87c124504d4d4699185ef2652e201f: Neither blocks nor log segments found. Creating new log.
I20260811 19:02:59.933142 32100 log.cc:824] T 3543092451244c62b56ac491d3c3afbe P 5dcf1b86861b46ea8173b48801406752: Log is configured to *not* fsync() on all Append() calls
I20260811 19:02:59.933954 32101 log.cc:824] T 3543092451244c62b56ac491d3c3afbe P 9f87c124504d4d4699185ef2652e201f: Log is configured to *not* fsync() on all Append() calls
I20260811 19:02:59.936625 32100 tablet_bootstrap.cc:492] T 3543092451244c62b56ac491d3c3afbe P 5dcf1b86861b46ea8173b48801406752: No bootstrap required, opened a new log
I20260811 19:02:59.936640 32101 tablet_bootstrap.cc:492] T 3543092451244c62b56ac491d3c3afbe P 9f87c124504d4d4699185ef2652e201f: No bootstrap required, opened a new log
I20260811 19:02:59.936897 32101 ts_tablet_manager.cc:1397] T 3543092451244c62b56ac491d3c3afbe P 9f87c124504d4d4699185ef2652e201f: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.006s
I20260811 19:02:59.936919 32100 ts_tablet_manager.cc:1397] T 3543092451244c62b56ac491d3c3afbe P 5dcf1b86861b46ea8173b48801406752: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260811 19:02:59.943707 32099 raft_consensus.cc:348] T 3543092451244c62b56ac491d3c3afbe P 3be6873b0693445f8d35457b87bab346 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3be6873b0693445f8d35457b87bab346" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 39793 } } peers { permanent_uuid: "9f87c124504d4d4699185ef2652e201f" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 38943 } } peers { permanent_uuid: "5dcf1b86861b46ea8173b48801406752" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33947 } }
I20260811 19:02:59.944221 32099 raft_consensus.cc:374] T 3543092451244c62b56ac491d3c3afbe P 3be6873b0693445f8d35457b87bab346 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:02:59.944332 32099 raft_consensus.cc:729] T 3543092451244c62b56ac491d3c3afbe P 3be6873b0693445f8d35457b87bab346 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3be6873b0693445f8d35457b87bab346, State: Initialized, Role: FOLLOWER
I20260811 19:02:59.944329 32101 raft_consensus.cc:348] T 3543092451244c62b56ac491d3c3afbe P 9f87c124504d4d4699185ef2652e201f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3be6873b0693445f8d35457b87bab346" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 39793 } } peers { permanent_uuid: "9f87c124504d4d4699185ef2652e201f" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 38943 } } peers { permanent_uuid: "5dcf1b86861b46ea8173b48801406752" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33947 } }
I20260811 19:02:59.944773 32101 raft_consensus.cc:374] T 3543092451244c62b56ac491d3c3afbe P 9f87c124504d4d4699185ef2652e201f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:02:59.944871 32101 raft_consensus.cc:729] T 3543092451244c62b56ac491d3c3afbe P 9f87c124504d4d4699185ef2652e201f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f87c124504d4d4699185ef2652e201f, State: Initialized, Role: FOLLOWER
I20260811 19:02:59.944968 32099 consensus_queue.cc:260] T 3543092451244c62b56ac491d3c3afbe P 3be6873b0693445f8d35457b87bab346 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3be6873b0693445f8d35457b87bab346" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 39793 } } peers { permanent_uuid: "9f87c124504d4d4699185ef2652e201f" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 38943 } } peers { permanent_uuid: "5dcf1b86861b46ea8173b48801406752" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33947 } }
I20260811 19:02:59.945487 32101 consensus_queue.cc:260] T 3543092451244c62b56ac491d3c3afbe P 9f87c124504d4d4699185ef2652e201f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3be6873b0693445f8d35457b87bab346" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 39793 } } peers { permanent_uuid: "9f87c124504d4d4699185ef2652e201f" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 38943 } } peers { permanent_uuid: "5dcf1b86861b46ea8173b48801406752" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33947 } }
I20260811 19:02:59.946305 32067 heartbeater.cc:503] Master 127.29.112.190:34291 was elected leader, sending a full tablet report...
I20260811 19:02:59.946873 32099 ts_tablet_manager.cc:1428] T 3543092451244c62b56ac491d3c3afbe P 3be6873b0693445f8d35457b87bab346: Time spent starting tablet: real 0.014s user 0.016s sys 0.000s
I20260811 19:02:59.947348 32100 raft_consensus.cc:348] T 3543092451244c62b56ac491d3c3afbe P 5dcf1b86861b46ea8173b48801406752 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3be6873b0693445f8d35457b87bab346" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 39793 } } peers { permanent_uuid: "9f87c124504d4d4699185ef2652e201f" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 38943 } } peers { permanent_uuid: "5dcf1b86861b46ea8173b48801406752" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33947 } }
I20260811 19:02:59.947745 32100 raft_consensus.cc:374] T 3543092451244c62b56ac491d3c3afbe P 5dcf1b86861b46ea8173b48801406752 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:02:59.947815 32100 raft_consensus.cc:729] T 3543092451244c62b56ac491d3c3afbe P 5dcf1b86861b46ea8173b48801406752 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5dcf1b86861b46ea8173b48801406752, State: Initialized, Role: FOLLOWER
I20260811 19:02:59.948391 32100 consensus_queue.cc:260] T 3543092451244c62b56ac491d3c3afbe P 5dcf1b86861b46ea8173b48801406752 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3be6873b0693445f8d35457b87bab346" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 39793 } } peers { permanent_uuid: "9f87c124504d4d4699185ef2652e201f" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 38943 } } peers { permanent_uuid: "5dcf1b86861b46ea8173b48801406752" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33947 } }
I20260811 19:02:59.949293 32101 ts_tablet_manager.cc:1428] T 3543092451244c62b56ac491d3c3afbe P 9f87c124504d4d4699185ef2652e201f: Time spent starting tablet: real 0.012s user 0.011s sys 0.002s
I20260811 19:02:59.950935 31833 heartbeater.cc:503] Master 127.29.112.190:34291 was elected leader, sending a full tablet report...
I20260811 19:02:59.951084 32100 ts_tablet_manager.cc:1428] T 3543092451244c62b56ac491d3c3afbe P 5dcf1b86861b46ea8173b48801406752: Time spent starting tablet: real 0.014s user 0.015s sys 0.001s
W20260811 19:03:00.000626 31586 tablet.cc:2369] T 3543092451244c62b56ac491d3c3afbe P 9f87c124504d4d4699185ef2652e201f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 19:03:00.031725 32106 raft_consensus.cc:482] T 3543092451244c62b56ac491d3c3afbe P 3be6873b0693445f8d35457b87bab346 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:03:00.032140 32106 raft_consensus.cc:504] T 3543092451244c62b56ac491d3c3afbe P 3be6873b0693445f8d35457b87bab346 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3be6873b0693445f8d35457b87bab346" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 39793 } } peers { permanent_uuid: "9f87c124504d4d4699185ef2652e201f" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 38943 } } peers { permanent_uuid: "5dcf1b86861b46ea8173b48801406752" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33947 } }
I20260811 19:03:00.034075 32106 leader_election.cc:288] T 3543092451244c62b56ac491d3c3afbe P 3be6873b0693445f8d35457b87bab346 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9f87c124504d4d4699185ef2652e201f (127.29.112.129:38943), 5dcf1b86861b46ea8173b48801406752 (127.29.112.130:33947)
I20260811 19:03:00.041770 32094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:00.034702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.131:60647 (local address 127.29.112.130:33947)
0811 19:03:00.034897 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:00.034904 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:00.035540 (+ 636us) server_negotiation.cc:407] Connection header received
0811 19:03:00.035764 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:00.035770 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:00.035853 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:00.035953 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:03:00.036609 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:00.037664 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:00.040019 (+ 2355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:00.040782 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:00.040957 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:00.041027 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:00.041361 (+ 334us) server_negotiation.cc:299] Negotiation successful
0811 19:03:00.041510 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 19:03:00.041837 32070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:00.034301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.129:38943 (local address 127.29.112.131:42755)
0811 19:03:00.034972 (+ 671us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:00.034999 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 19:03:00.035150 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:03:00.036206 (+ 1056us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:03:00.036210 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:03:00.036232 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:03:00.036439 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:03:00.036448 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:03:00.037463 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:03:00.037468 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:03:00.039595 (+ 2127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:03:00.039610 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:03:00.041125 (+ 1515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:03:00.041131 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:03:00.041334 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:00.041363 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 19:03:00.041476 (+ 113us) client_negotiation.cc:240] Negotiation successful
0811 19:03:00.041590 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":517,"mutex_wait_us":1}
I20260811 19:03:00.041874 32113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:00.034554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.130:33947 (local address 127.29.112.131:60647)
0811 19:03:00.035404 (+ 850us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:00.035423 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 19:03:00.035561 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:03:00.035994 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:03:00.036000 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:03:00.036025 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:03:00.036413 (+ 388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:03:00.036423 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:03:00.037821 (+ 1398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:03:00.037827 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:03:00.039828 (+ 2001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:03:00.039838 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:03:00.040954 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:03:00.040960 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:03:00.041180 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:00.041218 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 19:03:00.041312 (+ 94us) client_negotiation.cc:240] Negotiation successful
0811 19:03:00.041462 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":739,"mutex_wait_us":28,"thread_start_us":477,"threads_started":1}
I20260811 19:03:00.042728 32098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:00.035561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.112.131:42755 (local address 127.29.112.129:38943)
0811 19:03:00.035755 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:00.035761 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:00.035779 (+ 18us) server_negotiation.cc:407] Connection header received
0811 19:03:00.035859 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:00.035863 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:00.035950 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:00.036028 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:03:00.036601 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:00.037313 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:00.039771 (+ 2458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:00.040985 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:00.041129 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:00.041192 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:00.042174 (+ 982us) server_negotiation.cc:299] Negotiation successful
0811 19:03:00.042312 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 19:03:00.043716 31755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3543092451244c62b56ac491d3c3afbe" candidate_uuid: "3be6873b0693445f8d35457b87bab346" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5dcf1b86861b46ea8173b48801406752" is_pre_election: true
I20260811 19:03:00.044231 31513 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3543092451244c62b56ac491d3c3afbe" candidate_uuid: "3be6873b0693445f8d35457b87bab346" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f87c124504d4d4699185ef2652e201f" is_pre_election: true
I20260811 19:03:00.044663 31755 raft_consensus.cc:2454] T 3543092451244c62b56ac491d3c3afbe P 5dcf1b86861b46ea8173b48801406752 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3be6873b0693445f8d35457b87bab346 in term 0.
I20260811 19:03:00.045038 31513 raft_consensus.cc:2454] T 3543092451244c62b56ac491d3c3afbe P 9f87c124504d4d4699185ef2652e201f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3be6873b0693445f8d35457b87bab346 in term 0.
I20260811 19:03:00.045883 31878 leader_election.cc:302] T 3543092451244c62b56ac491d3c3afbe P 3be6873b0693445f8d35457b87bab346 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3be6873b0693445f8d35457b87bab346, 5dcf1b86861b46ea8173b48801406752; no voters:
I20260811 19:03:00.046362 32106 raft_consensus.cc:2781] T 3543092451244c62b56ac491d3c3afbe P 3be6873b0693445f8d35457b87bab346 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 19:03:00.046590 32106 raft_consensus.cc:482] T 3543092451244c62b56ac491d3c3afbe P 3be6873b0693445f8d35457b87bab346 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 19:03:00.046736 32106 raft_consensus.cc:3037] T 3543092451244c62b56ac491d3c3afbe P 3be6873b0693445f8d35457b87bab346 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:03:00.049219 32106 raft_consensus.cc:504] T 3543092451244c62b56ac491d3c3afbe P 3be6873b0693445f8d35457b87bab346 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3be6873b0693445f8d35457b87bab346" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 39793 } } peers { permanent_uuid: "9f87c124504d4d4699185ef2652e201f" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 38943 } } peers { permanent_uuid: "5dcf1b86861b46ea8173b48801406752" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33947 } }
I20260811 19:03:00.049876 32106 leader_election.cc:288] T 3543092451244c62b56ac491d3c3afbe P 3be6873b0693445f8d35457b87bab346 [CANDIDATE]: Term 1 election: Requested vote from peers 9f87c124504d4d4699185ef2652e201f (127.29.112.129:38943), 5dcf1b86861b46ea8173b48801406752 (127.29.112.130:33947)
W20260811 19:03:00.050361 31839 tablet.cc:2369] T 3543092451244c62b56ac491d3c3afbe P 5dcf1b86861b46ea8173b48801406752: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 19:03:00.050486 31755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3543092451244c62b56ac491d3c3afbe" candidate_uuid: "3be6873b0693445f8d35457b87bab346" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5dcf1b86861b46ea8173b48801406752"
I20260811 19:03:00.050477 31513 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3543092451244c62b56ac491d3c3afbe" candidate_uuid: "3be6873b0693445f8d35457b87bab346" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f87c124504d4d4699185ef2652e201f"
I20260811 19:03:00.050755 31755 raft_consensus.cc:3037] T 3543092451244c62b56ac491d3c3afbe P 5dcf1b86861b46ea8173b48801406752 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:03:00.050992 31513 raft_consensus.cc:3037] T 3543092451244c62b56ac491d3c3afbe P 9f87c124504d4d4699185ef2652e201f [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:03:00.053418 31513 raft_consensus.cc:2454] T 3543092451244c62b56ac491d3c3afbe P 9f87c124504d4d4699185ef2652e201f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3be6873b0693445f8d35457b87bab346 in term 1.
I20260811 19:03:00.053493 31755 raft_consensus.cc:2454] T 3543092451244c62b56ac491d3c3afbe P 5dcf1b86861b46ea8173b48801406752 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3be6873b0693445f8d35457b87bab346 in term 1.
I20260811 19:03:00.053951 31877 leader_election.cc:302] T 3543092451244c62b56ac491d3c3afbe P 3be6873b0693445f8d35457b87bab346 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3be6873b0693445f8d35457b87bab346, 9f87c124504d4d4699185ef2652e201f; no voters:
I20260811 19:03:00.054306 32106 raft_consensus.cc:2781] T 3543092451244c62b56ac491d3c3afbe P 3be6873b0693445f8d35457b87bab346 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:03:00.055087 32106 raft_consensus.cc:686] T 3543092451244c62b56ac491d3c3afbe P 3be6873b0693445f8d35457b87bab346 [term 1 LEADER]: Becoming Leader. State: Replica: 3be6873b0693445f8d35457b87bab346, State: Running, Role: LEADER
I20260811 19:03:00.055848 32106 consensus_queue.cc:237] T 3543092451244c62b56ac491d3c3afbe P 3be6873b0693445f8d35457b87bab346 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3be6873b0693445f8d35457b87bab346" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 39793 } } peers { permanent_uuid: "9f87c124504d4d4699185ef2652e201f" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 38943 } } peers { permanent_uuid: "5dcf1b86861b46ea8173b48801406752" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33947 } }
I20260811 19:03:00.062172 31261 catalog_manager.cc:5495] T 3543092451244c62b56ac491d3c3afbe P 3be6873b0693445f8d35457b87bab346 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3be6873b0693445f8d35457b87bab346 (127.29.112.131). New cstate: current_term: 1 leader_uuid: "3be6873b0693445f8d35457b87bab346" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3be6873b0693445f8d35457b87bab346" member_type: VOTER last_known_addr { host: "127.29.112.131" port: 39793 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9f87c124504d4d4699185ef2652e201f" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5dcf1b86861b46ea8173b48801406752" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33947 } health_report { overall_health: UNKNOWN } } }
W20260811 19:03:00.070115 32068 tablet.cc:2369] T 3543092451244c62b56ac491d3c3afbe P 3be6873b0693445f8d35457b87bab346: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 19:03:00.100875 32092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:00.093560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34016 (local address 127.29.112.131:39793)
0811 19:03:00.093770 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:00.093775 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:00.093794 (+ 19us) server_negotiation.cc:407] Connection header received
0811 19:03:00.093902 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:00.093906 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:00.094011 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:00.094158 (+ 147us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:03:00.094745 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:00.095778 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:00.097414 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:00.098030 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:00.098132 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:00.098841 (+ 709us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:03:00.099927 (+ 1086us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:03:00.100324 (+ 397us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:00.100406 (+ 82us) server_negotiation.cc:299] Negotiation successful
0811 19:03:00.100529 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260811 19:03:00.115828 31513 raft_consensus.cc:1261] T 3543092451244c62b56ac491d3c3afbe P 9f87c124504d4d4699185ef2652e201f [term 1 FOLLOWER]: Refusing update from remote peer 3be6873b0693445f8d35457b87bab346: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:03:00.116966 31755 raft_consensus.cc:1261] T 3543092451244c62b56ac491d3c3afbe P 5dcf1b86861b46ea8173b48801406752 [term 1 FOLLOWER]: Refusing update from remote peer 3be6873b0693445f8d35457b87bab346: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:03:00.117724 32106 consensus_queue.cc:1035] T 3543092451244c62b56ac491d3c3afbe P 3be6873b0693445f8d35457b87bab346 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f87c124504d4d4699185ef2652e201f" member_type: VOTER last_known_addr { host: "127.29.112.129" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:03:00.118112 32114 consensus_queue.cc:1035] T 3543092451244c62b56ac491d3c3afbe P 3be6873b0693445f8d35457b87bab346 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5dcf1b86861b46ea8173b48801406752" member_type: VOTER last_known_addr { host: "127.29.112.130" port: 33947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:03:00.145181 31259 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49650:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:03:00.145476 31259 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49650:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:03:00.149951 31259 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 42b24f555a064b248a9bd4a01c808016: Sending DeleteTablet for 3 replicas of tablet 3543092451244c62b56ac491d3c3afbe
I20260811 19:03:00.151645 31724 tablet_service.cc:1514] Processing DeleteTablet for tablet 3543092451244c62b56ac491d3c3afbe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:03:00 UTC) from {username='slave'} at 127.0.0.1:44270
I20260811 19:03:00.151667 31956 tablet_service.cc:1514] Processing DeleteTablet for tablet 3543092451244c62b56ac491d3c3afbe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:03:00 UTC) from {username='slave'} at 127.0.0.1:34010
I20260811 19:03:00.152652 31477 tablet_service.cc:1514] Processing DeleteTablet for tablet 3543092451244c62b56ac491d3c3afbe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:03:00 UTC) from {username='slave'} at 127.0.0.1:41566
I20260811 19:03:00.153267 32145 tablet_replica.cc:330] T 3543092451244c62b56ac491d3c3afbe P 3be6873b0693445f8d35457b87bab346: stopping tablet replica
I20260811 19:03:00.153784 32145 raft_consensus.cc:2229] T 3543092451244c62b56ac491d3c3afbe P 3be6873b0693445f8d35457b87bab346 [term 1 LEADER]: Raft consensus shutting down.
I20260811 19:03:00.154139 32146 tablet_replica.cc:330] T 3543092451244c62b56ac491d3c3afbe P 9f87c124504d4d4699185ef2652e201f: stopping tablet replica
I20260811 19:03:00.154560 32145 raft_consensus.cc:2258] T 3543092451244c62b56ac491d3c3afbe P 3be6873b0693445f8d35457b87bab346 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:03:00.154783 32146 raft_consensus.cc:2229] T 3543092451244c62b56ac491d3c3afbe P 9f87c124504d4d4699185ef2652e201f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:03:00.154979 32144 tablet_replica.cc:330] T 3543092451244c62b56ac491d3c3afbe P 5dcf1b86861b46ea8173b48801406752: stopping tablet replica
I20260811 19:03:00.155107 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 31365
I20260811 19:03:00.155386 32144 raft_consensus.cc:2229] T 3543092451244c62b56ac491d3c3afbe P 5dcf1b86861b46ea8173b48801406752 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:03:00.155476 32146 raft_consensus.cc:2258] T 3543092451244c62b56ac491d3c3afbe P 9f87c124504d4d4699185ef2652e201f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:03:00.155857 32144 raft_consensus.cc:2258] T 3543092451244c62b56ac491d3c3afbe P 5dcf1b86861b46ea8173b48801406752 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:03:00.157970 32146 ts_tablet_manager.cc:1905] T 3543092451244c62b56ac491d3c3afbe P 9f87c124504d4d4699185ef2652e201f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:03:00.158105 32145 ts_tablet_manager.cc:1905] T 3543092451244c62b56ac491d3c3afbe P 3be6873b0693445f8d35457b87bab346: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:03:00.165783 32144 ts_tablet_manager.cc:1905] T 3543092451244c62b56ac491d3c3afbe P 5dcf1b86861b46ea8173b48801406752: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:03:00.166025 32145 ts_tablet_manager.cc:1918] T 3543092451244c62b56ac491d3c3afbe P 3be6873b0693445f8d35457b87bab346: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:03:00.166218 32145 log.cc:1196] T 3543092451244c62b56ac491d3c3afbe P 3be6873b0693445f8d35457b87bab346: Deleting WAL directory at /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474615205622-30146-0/minicluster-data/ts-2/wal/wals/3543092451244c62b56ac491d3c3afbe
I20260811 19:03:00.166801 32145 ts_tablet_manager.cc:1939] T 3543092451244c62b56ac491d3c3afbe P 3be6873b0693445f8d35457b87bab346: Deleting consensus metadata
I20260811 19:03:00.166975 32146 ts_tablet_manager.cc:1918] T 3543092451244c62b56ac491d3c3afbe P 9f87c124504d4d4699185ef2652e201f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:03:00.167213 32146 log.cc:1196] T 3543092451244c62b56ac491d3c3afbe P 9f87c124504d4d4699185ef2652e201f: Deleting WAL directory at /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474615205622-30146-0/minicluster-data/ts-0/wal/wals/3543092451244c62b56ac491d3c3afbe
I20260811 19:03:00.167800 32146 ts_tablet_manager.cc:1939] T 3543092451244c62b56ac491d3c3afbe P 9f87c124504d4d4699185ef2652e201f: Deleting consensus metadata
I20260811 19:03:00.168785 31228 catalog_manager.cc:4841] TS 3be6873b0693445f8d35457b87bab346 (127.29.112.131:39793): tablet 3543092451244c62b56ac491d3c3afbe (table test-table [id=b044650450514c1eaa5536dc8ac8f847]) successfully deleted
I20260811 19:03:00.169620 31230 catalog_manager.cc:4841] TS 9f87c124504d4d4699185ef2652e201f (127.29.112.129:38943): tablet 3543092451244c62b56ac491d3c3afbe (table test-table [id=b044650450514c1eaa5536dc8ac8f847]) successfully deleted
I20260811 19:03:00.175412 32144 ts_tablet_manager.cc:1918] T 3543092451244c62b56ac491d3c3afbe P 5dcf1b86861b46ea8173b48801406752: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:03:00.175611 32144 log.cc:1196] T 3543092451244c62b56ac491d3c3afbe P 5dcf1b86861b46ea8173b48801406752: Deleting WAL directory at /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474615205622-30146-0/minicluster-data/ts-1/wal/wals/3543092451244c62b56ac491d3c3afbe
I20260811 19:03:00.176254 32144 ts_tablet_manager.cc:1939] T 3543092451244c62b56ac491d3c3afbe P 5dcf1b86861b46ea8173b48801406752: Deleting consensus metadata
I20260811 19:03:00.178066 31231 catalog_manager.cc:4841] TS 5dcf1b86861b46ea8173b48801406752 (127.29.112.130:33947): tablet 3543092451244c62b56ac491d3c3afbe (table test-table [id=b044650450514c1eaa5536dc8ac8f847]) successfully deleted
I20260811 19:03:00.179085 32098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:00.156743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41582 (local address 127.29.112.129:38943)
0811 19:03:00.156918 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:00.156924 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:00.171149 (+ 14225us) server_negotiation.cc:407] Connection header received
0811 19:03:00.171348 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:00.171356 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:00.171473 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:00.171604 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:00.172087 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:00.173216 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:00.175189 (+ 1973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:00.175822 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:00.175955 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:00.177307 (+ 1352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:03:00.177318 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:03:00.177336 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:03:00.178189 (+ 853us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:03:00.178249 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:03:00.178276 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:03:00.178395 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:00.178572 (+ 177us) server_negotiation.cc:299] Negotiation successful
0811 19:03:00.178685 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 19:03:00.179807 31580 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:03:00.313238 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 31365
I20260811 19:03:00.352224 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 31591
I20260811 19:03:00.361486 32094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:00.353477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44280 (local address 127.29.112.130:33947)
0811 19:03:00.353708 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:00.353715 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:00.353738 (+ 23us) server_negotiation.cc:407] Connection header received
0811 19:03:00.353808 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:00.353814 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:00.353926 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:00.354044 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:00.355042 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:00.356212 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:00.357632 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:00.358287 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:00.358419 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:00.359643 (+ 1224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:03:00.359655 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:03:00.359673 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:03:00.360497 (+ 824us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:03:00.360557 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:03:00.360584 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:03:00.360701 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:00.360988 (+ 287us) server_negotiation.cc:299] Negotiation successful
0811 19:03:00.361105 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260811 19:03:00.362367 31823 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:03:00.459558 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 31591
I20260811 19:03:00.491110 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 31845
I20260811 19:03:00.501506 32092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:00.492196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34028 (local address 127.29.112.131:39793)
0811 19:03:00.492445 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:00.492457 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:00.492483 (+ 26us) server_negotiation.cc:407] Connection header received
0811 19:03:00.492672 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:00.492677 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:00.492778 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:00.492895 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:00.493351 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:00.494157 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:00.496136 (+ 1979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:00.496711 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:00.496884 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:00.499211 (+ 2327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:03:00.499223 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:03:00.499243 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:03:00.500155 (+ 912us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:03:00.500232 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:03:00.500267 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:03:00.500471 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:00.500751 (+ 280us) server_negotiation.cc:299] Negotiation successful
0811 19:03:00.500918 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 19:03:00.502717 32053 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:03:00.640754 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 31845
I20260811 19:03:00.673532 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 31199
I20260811 19:03:00.675400 31317 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:03:00.793736 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 31199
2026-08-11T19:03:00Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (3403 ms)
[----------] 6 tests from EncryptionPolicyTest (28594 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474615205622-30146-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:03:00 dist-test-slave-h6n0 krb5kdc[32156](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:03:00 dist-test-slave-h6n0 krb5kdc[32156](info): set up 2 sockets
Aug 11 19:03:00 dist-test-slave-h6n0 krb5kdc[32156](info): commencing operation
krb5kdc: starting...
W20260811 19:03:02.350593 30146 mini_kdc.cc:121] Time spent starting KDC: real 1.495s user 0.000s sys 0.057s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474615205622-30146-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474615205622-30146-0/krb5kdc/test-user.keytab.
Aug 11 19:03:02 dist-test-slave-h6n0 krb5kdc[32156](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474982, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:03:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:03:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474615205622-30146-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474615205622-30146-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 19:03:02.723483 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474615205622-30146-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474615205622-30146-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474615205622-30146-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45733
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474615205622-30146-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45733
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:03:03.078620 32185 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:03:03.078984 32185 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:03:03.079161 32185 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:03:03.086441 32185 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:03:03.086863 32185 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:03:03.087049 32185 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:03:03.087177 32185 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:03:03.087435 32185 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:03:03.097846 32185 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474615205622-30146-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474615205622-30146-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45733
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474615205622-30146-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45733
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474615205622-30146-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:03:03.099279 32185 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:03:03.100816 32185 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:03:03.113914 32193 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:03:03.113940 32194 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:03:03.113956 32196 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:03:03.114769 32185 server_base.cc:1034] running on GCE node
I20260811 19:03:03.116034 32185 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:03:03.117523 32185 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:03:03.118733 32185 hybrid_clock.cc:648] HybridClock initialized: now 1786474983118688 us; error 38 us; skew 500 ppm
I20260811 19:03:03.119124 32185 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:03:03 dist-test-slave-h6n0 krb5kdc[32156](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474983, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:03:03.122676 32185 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 19:03:03.125015 32185 webserver.cc:460] Webserver started at http://127.0.0.1:39623/ using document root <none> and password file <none>
I20260811 19:03:03.126236 32185 fs_manager.cc:359] Metadata directory not provided
I20260811 19:03:03.126449 32185 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:03:03.126823 32185 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:03:03.130038 32185 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474615205622-30146-0/minicluster-data/master-0/data/instance:
uuid: "c8768e873e0e48668b084b320f4347bc"
format_stamp: "Formatted at 2026-08-11 19:03:03 on dist-test-slave-h6n0"
server_key: "bc883f191a2e6e8036f67cbd41971b03"
server_key_iv: "861f7f653e85b3d2ff20d816a1962f65"
server_key_version: "encryptionkey@0"
I20260811 19:03:03.130916 32185 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474615205622-30146-0/minicluster-data/master-0/wal/instance:
uuid: "c8768e873e0e48668b084b320f4347bc"
format_stamp: "Formatted at 2026-08-11 19:03:03 on dist-test-slave-h6n0"
server_key: "bc883f191a2e6e8036f67cbd41971b03"
server_key_iv: "861f7f653e85b3d2ff20d816a1962f65"
server_key_version: "encryptionkey@0"
I20260811 19:03:03.137624 32185 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 19:03:03.142314 32208 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:03:03.144282 32185 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 19:03:03.144519 32185 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474615205622-30146-0/minicluster-data/master-0/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474615205622-30146-0/minicluster-data/master-0/wal
uuid: "c8768e873e0e48668b084b320f4347bc"
format_stamp: "Formatted at 2026-08-11 19:03:03 on dist-test-slave-h6n0"
server_key: "bc883f191a2e6e8036f67cbd41971b03"
server_key_iv: "861f7f653e85b3d2ff20d816a1962f65"
server_key_version: "encryptionkey@0"
I20260811 19:03:03.144758 32185 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474615205622-30146-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474615205622-30146-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474615205622-30146-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:03:03.165055 32185 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:03:03.171005 32185 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:03:03.171491 32185 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:03:03.196367 32185 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45733
I20260811 19:03:03.196447 32295 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45733 every 8 connection(s)
I20260811 19:03:03.198521 32185 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
I20260811 19:03:03.202858 32298 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:03:03.208437 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 32185
I20260811 19:03:03.208693 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474615205622-30146-0/minicluster-data/master-0/wal/instance
I20260811 19:03:03.209393 30146 external_mini_cluster.cc:1428] Setting key 96a21533300444aa1cdc56976bbd3129
I20260811 19:03:03.214280 32298 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8768e873e0e48668b084b320f4347bc: Bootstrap starting.
I20260811 19:03:03.219089 32298 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c8768e873e0e48668b084b320f4347bc: Neither blocks nor log segments found. Creating new log.
I20260811 19:03:03.220263 32298 log.cc:824] T 00000000000000000000000000000000 P c8768e873e0e48668b084b320f4347bc: Log is configured to *not* fsync() on all Append() calls
I20260811 19:03:03.224232 32298 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8768e873e0e48668b084b320f4347bc: No bootstrap required, opened a new log
Aug 11 19:03:03 dist-test-slave-h6n0 krb5kdc[32156](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474982, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 19:03:03.232955 32298 raft_consensus.cc:348] T 00000000000000000000000000000000 P c8768e873e0e48668b084b320f4347bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8768e873e0e48668b084b320f4347bc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45733 } }
I20260811 19:03:03.233397 32298 raft_consensus.cc:374] T 00000000000000000000000000000000 P c8768e873e0e48668b084b320f4347bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:03:03.233518 32298 raft_consensus.cc:729] T 00000000000000000000000000000000 P c8768e873e0e48668b084b320f4347bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8768e873e0e48668b084b320f4347bc, State: Initialized, Role: FOLLOWER
I20260811 19:03:03.234143 32298 consensus_queue.cc:260] T 00000000000000000000000000000000 P c8768e873e0e48668b084b320f4347bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8768e873e0e48668b084b320f4347bc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45733 } }
I20260811 19:03:03.234402 32298 raft_consensus.cc:388] T 00000000000000000000000000000000 P c8768e873e0e48668b084b320f4347bc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:03:03.234623 32298 raft_consensus.cc:482] T 00000000000000000000000000000000 P c8768e873e0e48668b084b320f4347bc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:03:03.234831 32298 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c8768e873e0e48668b084b320f4347bc [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:03:03.237329 32301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:03.211508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44537 (local address 127.0.0.1:45733)
0811 19:03:03.212441 (+ 933us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:03.212460 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:03.212500 (+ 40us) server_negotiation.cc:407] Connection header received
0811 19:03:03.214173 (+ 1673us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:03.214193 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:03.214670 (+ 477us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:03.214979 (+ 309us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:03.217929 (+ 2950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:03.220041 (+ 2112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:03.222659 (+ 2618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:03.223400 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:03.223553 (+ 153us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:03.228486 (+ 4933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:03:03.228494 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:03:03.228509 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:03:03.228549 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:03:03.231952 (+ 3403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:03.232884 (+ 932us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:03.232890 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:03.232896 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:03.233119 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:03.233674 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:03.233679 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:03.233688 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:03.234383 (+ 695us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:03:03.234748 (+ 365us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:03.235332 (+ 584us) server_negotiation.cc:299] Negotiation successful
0811 19:03:03.235707 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":703,"thread_start_us":459,"threads_started":1}
I20260811 19:03:03.238364 32298 raft_consensus.cc:504] T 00000000000000000000000000000000 P c8768e873e0e48668b084b320f4347bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8768e873e0e48668b084b320f4347bc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45733 } }
I20260811 19:03:03.238922 32298 leader_election.cc:302] T 00000000000000000000000000000000 P c8768e873e0e48668b084b320f4347bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c8768e873e0e48668b084b320f4347bc; no voters:
I20260811 19:03:03.239929 32298 leader_election.cc:288] T 00000000000000000000000000000000 P c8768e873e0e48668b084b320f4347bc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:03:03.240095 32304 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c8768e873e0e48668b084b320f4347bc [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:03:03.241523 32304 raft_consensus.cc:686] T 00000000000000000000000000000000 P c8768e873e0e48668b084b320f4347bc [term 1 LEADER]: Becoming Leader. State: Replica: c8768e873e0e48668b084b320f4347bc, State: Running, Role: LEADER
I20260811 19:03:03.242143 30146 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 19:03:03.242246 32298 sys_catalog.cc:564] T 00000000000000000000000000000000 P c8768e873e0e48668b084b320f4347bc [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:03:03.242158 32304 consensus_queue.cc:237] T 00000000000000000000000000000000 P c8768e873e0e48668b084b320f4347bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8768e873e0e48668b084b320f4347bc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45733 } }
I20260811 19:03:03.244827 32309 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8768e873e0e48668b084b320f4347bc [sys.catalog]: SysCatalogTable state changed. Reason: New leader c8768e873e0e48668b084b320f4347bc. Latest consensus state: current_term: 1 leader_uuid: "c8768e873e0e48668b084b320f4347bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8768e873e0e48668b084b320f4347bc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45733 } } }
I20260811 19:03:03.245220 32309 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8768e873e0e48668b084b320f4347bc [sys.catalog]: This master's current role is: LEADER
I20260811 19:03:03.245779 32308 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8768e873e0e48668b084b320f4347bc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c8768e873e0e48668b084b320f4347bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8768e873e0e48668b084b320f4347bc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45733 } } }
I20260811 19:03:03.246101 32308 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8768e873e0e48668b084b320f4347bc [sys.catalog]: This master's current role is: LEADER
I20260811 19:03:03.249346 32312 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:03:03.257097 32312 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:03:03.271857 32312 catalog_manager.cc:1347] Generated new cluster ID: 600cca24eb124058b82472dbcdd5af60
I20260811 19:03:03.271988 32312 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:03:03.277153 32301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:03.252821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49201 (local address 127.0.0.1:45733)
0811 19:03:03.253052 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:03.253061 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:03.253713 (+ 652us) server_negotiation.cc:407] Connection header received
0811 19:03:03.254048 (+ 335us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:03.254060 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:03.254172 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:03.254264 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:03.256983 (+ 2719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:03.258892 (+ 1909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:03.261698 (+ 2806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:03.262968 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:03.263153 (+ 185us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:03.268548 (+ 5395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:03:03.268560 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:03:03.268564 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:03:03.268610 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:03:03.272236 (+ 3626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:03.273909 (+ 1673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:03.273920 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:03.273924 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:03.274170 (+ 246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:03.274966 (+ 796us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:03.274972 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:03.274976 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:03.275517 (+ 541us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:03:03.275849 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:03.276429 (+ 580us) server_negotiation.cc:299] Negotiation successful
0811 19:03:03.276701 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 19:03:03.292012 32312 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:03:03.293449 32312 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:03:03.303143 32312 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c8768e873e0e48668b084b320f4347bc: Generated new TSK 0
I20260811 19:03:03.304670 32312 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 19:03:03.318255 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 32185
I20260811 19:03:03.319996 32288 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:03:03.419416 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 32185
W20260811 19:03:03.449499 32153 connection.cc:535] client connection to 127.0.0.1:45733 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45733): Connection reset by peer (error 104)
2026-08-11T19:03:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2616 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474615205622-30146-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:03:03 dist-test-slave-h6n0 krb5kdc[32355](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:03:03 dist-test-slave-h6n0 krb5kdc[32355](info): set up 2 sockets
Aug 11 19:03:03 dist-test-slave-h6n0 krb5kdc[32355](info): commencing operation
krb5kdc: starting...
W20260811 19:03:04.860888 30146 mini_kdc.cc:121] Time spent starting KDC: real 1.392s user 0.008s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474615205622-30146-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474615205622-30146-0/krb5kdc/test-user.keytab.
Aug 11 19:03:05 dist-test-slave-h6n0 krb5kdc[32355](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474985, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:03:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:03:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474615205622-30146-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474615205622-30146-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 19:03:05.205852 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474615205622-30146-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474615205622-30146-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474615205622-30146-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38673
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474615205622-30146-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38673
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:03:05.560094 32379 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:03:05.560469 32379 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:03:05.560653 32379 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:03:05.568802 32379 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:03:05.568988 32379 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:03:05.569098 32379 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:03:05.569212 32379 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:03:05.569291 32379 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:03:05.579655 32379 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474615205622-30146-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474615205622-30146-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38673
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474615205622-30146-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38673
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474615205622-30146-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:03:05.581080 32379 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:03:05.582489 32379 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:03:05.595320 32400 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:03:05.595477 32389 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:03:05.596447 32388 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:03:05.597548 32379 server_base.cc:1034] running on GCE node
I20260811 19:03:05.598433 32379 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:03:05.600124 32379 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:03:05.601339 32379 hybrid_clock.cc:648] HybridClock initialized: now 1786474985601310 us; error 41 us; skew 500 ppm
I20260811 19:03:05.601720 32379 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:03:05 dist-test-slave-h6n0 krb5kdc[32355](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474985, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:03:05.610605 32379 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 19:03:05.612929 32379 webserver.cc:460] Webserver started at http://127.0.0.1:44121/ using document root <none> and password file <none>
I20260811 19:03:05.613698 32379 fs_manager.cc:359] Metadata directory not provided
I20260811 19:03:05.613842 32379 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:03:05.614183 32379 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:03:05.617476 32379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474615205622-30146-0/minicluster-data/master-0/data/instance:
uuid: "d7f9bd8485f14aaaa40257ecfcd384a0"
format_stamp: "Formatted at 2026-08-11 19:03:05 on dist-test-slave-h6n0"
server_key: "45de497abcfc6752a1ee4147dfa474b8"
server_key_iv: "65651b4baa82e9ba1c64c3cc1fa7b588"
server_key_version: "encryptionkey@0"
I20260811 19:03:05.618283 32379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474615205622-30146-0/minicluster-data/master-0/wal/instance:
uuid: "d7f9bd8485f14aaaa40257ecfcd384a0"
format_stamp: "Formatted at 2026-08-11 19:03:05 on dist-test-slave-h6n0"
server_key: "45de497abcfc6752a1ee4147dfa474b8"
server_key_iv: "65651b4baa82e9ba1c64c3cc1fa7b588"
server_key_version: "encryptionkey@0"
I20260811 19:03:05.623775 32379 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260811 19:03:05.627879 32410 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:03:05.629295 32379 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 19:03:05.629516 32379 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474615205622-30146-0/minicluster-data/master-0/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474615205622-30146-0/minicluster-data/master-0/wal
uuid: "d7f9bd8485f14aaaa40257ecfcd384a0"
format_stamp: "Formatted at 2026-08-11 19:03:05 on dist-test-slave-h6n0"
server_key: "45de497abcfc6752a1ee4147dfa474b8"
server_key_iv: "65651b4baa82e9ba1c64c3cc1fa7b588"
server_key_version: "encryptionkey@0"
I20260811 19:03:05.629736 32379 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474615205622-30146-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474615205622-30146-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474615205622-30146-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:03:05.650151 32379 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:03:05.654479 32379 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:03:05.654808 32379 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:03:05.677596 32379 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38673
I20260811 19:03:05.677721 32501 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38673 every 8 connection(s)
I20260811 19:03:05.679667 32379 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
I20260811 19:03:05.684893 32502 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:03:05.689373 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 32379
I20260811 19:03:05.689582 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474615205622-30146-0/minicluster-data/master-0/wal/instance
I20260811 19:03:05.690188 30146 external_mini_cluster.cc:1428] Setting key 6ff4635096d64d788bc46b6df58e5e92
I20260811 19:03:05.697506 32502 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d7f9bd8485f14aaaa40257ecfcd384a0: Bootstrap starting.
I20260811 19:03:05.701406 32502 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d7f9bd8485f14aaaa40257ecfcd384a0: Neither blocks nor log segments found. Creating new log.
I20260811 19:03:05.702745 32502 log.cc:824] T 00000000000000000000000000000000 P d7f9bd8485f14aaaa40257ecfcd384a0: Log is configured to *not* fsync() on all Append() calls
I20260811 19:03:05.706204 32502 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d7f9bd8485f14aaaa40257ecfcd384a0: No bootstrap required, opened a new log
I20260811 19:03:05.711150 32509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:05.692590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37057 (local address 127.0.0.1:38673)
0811 19:03:05.693674 (+ 1084us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:05.693690 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:05.698372 (+ 4682us) server_negotiation.cc:407] Connection header received
0811 19:03:05.700654 (+ 2282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:05.700676 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:05.701059 (+ 383us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:05.701379 (+ 320us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:05.702022 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:05.703470 (+ 1448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:05.705360 (+ 1890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:05.706412 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:05.706641 (+ 229us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:05.707921 (+ 1280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:03:05.707932 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:03:05.707950 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:03:05.708762 (+ 812us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:03:05.708863 (+ 101us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:03:05.708898 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:03:05.709108 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:05.709489 (+ 381us) server_negotiation.cc:299] Negotiation successful
0811 19:03:05.709686 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":816,"thread_start_us":526,"threads_started":1}
I20260811 19:03:05.716048 32502 raft_consensus.cc:348] T 00000000000000000000000000000000 P d7f9bd8485f14aaaa40257ecfcd384a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7f9bd8485f14aaaa40257ecfcd384a0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38673 } }
I20260811 19:03:05.716532 32502 raft_consensus.cc:374] T 00000000000000000000000000000000 P d7f9bd8485f14aaaa40257ecfcd384a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:03:05.716677 32502 raft_consensus.cc:729] T 00000000000000000000000000000000 P d7f9bd8485f14aaaa40257ecfcd384a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7f9bd8485f14aaaa40257ecfcd384a0, State: Initialized, Role: FOLLOWER
I20260811 19:03:05.716717 30146 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 19:03:05.717342 32502 consensus_queue.cc:260] T 00000000000000000000000000000000 P d7f9bd8485f14aaaa40257ecfcd384a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7f9bd8485f14aaaa40257ecfcd384a0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38673 } }
I20260811 19:03:05.717628 32502 raft_consensus.cc:388] T 00000000000000000000000000000000 P d7f9bd8485f14aaaa40257ecfcd384a0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:03:05.717763 32502 raft_consensus.cc:482] T 00000000000000000000000000000000 P d7f9bd8485f14aaaa40257ecfcd384a0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:03:05.717926 32502 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d7f9bd8485f14aaaa40257ecfcd384a0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:03:05.721554 32502 raft_consensus.cc:504] T 00000000000000000000000000000000 P d7f9bd8485f14aaaa40257ecfcd384a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7f9bd8485f14aaaa40257ecfcd384a0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38673 } }
I20260811 19:03:05.722517 32502 leader_election.cc:302] T 00000000000000000000000000000000 P d7f9bd8485f14aaaa40257ecfcd384a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d7f9bd8485f14aaaa40257ecfcd384a0; no voters:
I20260811 19:03:05.723362 32502 leader_election.cc:288] T 00000000000000000000000000000000 P d7f9bd8485f14aaaa40257ecfcd384a0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:03:05.723517 32522 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d7f9bd8485f14aaaa40257ecfcd384a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:03:05.726359 32502 sys_catalog.cc:564] T 00000000000000000000000000000000 P d7f9bd8485f14aaaa40257ecfcd384a0 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:03:05.730151 32522 raft_consensus.cc:686] T 00000000000000000000000000000000 P d7f9bd8485f14aaaa40257ecfcd384a0 [term 1 LEADER]: Becoming Leader. State: Replica: d7f9bd8485f14aaaa40257ecfcd384a0, State: Running, Role: LEADER
I20260811 19:03:05.731066 32522 consensus_queue.cc:237] T 00000000000000000000000000000000 P d7f9bd8485f14aaaa40257ecfcd384a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7f9bd8485f14aaaa40257ecfcd384a0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38673 } }
I20260811 19:03:05.734788 32529 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7f9bd8485f14aaaa40257ecfcd384a0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d7f9bd8485f14aaaa40257ecfcd384a0. Latest consensus state: current_term: 1 leader_uuid: "d7f9bd8485f14aaaa40257ecfcd384a0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7f9bd8485f14aaaa40257ecfcd384a0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38673 } } }
I20260811 19:03:05.735289 32529 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7f9bd8485f14aaaa40257ecfcd384a0 [sys.catalog]: This master's current role is: LEADER
I20260811 19:03:05.736032 32526 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7f9bd8485f14aaaa40257ecfcd384a0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d7f9bd8485f14aaaa40257ecfcd384a0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7f9bd8485f14aaaa40257ecfcd384a0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38673 } } }
I20260811 19:03:05.736469 32526 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7f9bd8485f14aaaa40257ecfcd384a0 [sys.catalog]: This master's current role is: LEADER
I20260811 19:03:05.739487 32509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:05.723347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45609 (local address 127.0.0.1:38673)
0811 19:03:05.723658 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:05.723669 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:05.730099 (+ 6430us) server_negotiation.cc:407] Connection header received
0811 19:03:05.730438 (+ 339us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:05.730447 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:05.730592 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:05.730689 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:05.731692 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:05.732598 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:05.735249 (+ 2651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:05.735963 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:05.736069 (+ 106us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:05.738156 (+ 2087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:03:05.738167 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:03:05.738175 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:03:05.738209 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:03:05.738310 (+ 101us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:03:05.738341 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:03:05.738447 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:05.738925 (+ 478us) server_negotiation.cc:299] Negotiation successful
0811 19:03:05.739007 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260811 19:03:05.742091 32539 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:03:05.747973 32539 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:03:05.757697 32539 catalog_manager.cc:1347] Generated new cluster ID: 77e71f988aaa4df69f1fedacf29f739d
I20260811 19:03:05.757827 32539 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:03:05.768934 32539 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:03:05.770128 32539 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:03:05.779152 32539 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d7f9bd8485f14aaaa40257ecfcd384a0: Generated new TSK 0
I20260811 19:03:05.779981 32539 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 19:03:05.792948 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 32379
W20260811 19:03:05.794080 32494 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:37057
W20260811 19:03:05.794855 30146 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 32379: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 19:03:05.795053 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 32379
W20260811 19:03:05.822429 32352 connection.cc:535] client connection to 127.0.0.1:38673 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38673): Connection reset by peer (error 104)
2026-08-11T19:03:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (2372 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474615205622-30146-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:03:05 dist-test-slave-h6n0 krb5kdc[32558](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:03:05 dist-test-slave-h6n0 krb5kdc[32558](info): set up 2 sockets
Aug 11 19:03:05 dist-test-slave-h6n0 krb5kdc[32558](info): commencing operation
krb5kdc: starting...
W20260811 19:03:07.216498 30146 mini_kdc.cc:121] Time spent starting KDC: real 1.375s user 0.005s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474615205622-30146-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474615205622-30146-0/krb5kdc/test-user.keytab.
Aug 11 19:03:07 dist-test-slave-h6n0 krb5kdc[32558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474987, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:03:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:03:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474615205622-30146-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474615205622-30146-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 19:03:07.573797 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474615205622-30146-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474615205622-30146-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474615205622-30146-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38845
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474615205622-30146-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38845
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:03:07.941059 32584 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:03:07.941447 32584 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:03:07.941634 32584 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:03:07.950184 32584 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:03:07.950380 32584 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:03:07.950482 32584 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:03:07.950624 32584 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:03:07.950721 32584 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:03:07.961580 32584 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474615205622-30146-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474615205622-30146-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38845
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474615205622-30146-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38845
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474615205622-30146-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:03:07.963121 32584 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:03:07.964530 32584 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:03:07.979065 32592 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:03:07.979171 32590 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:03:07.979745 32599 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:03:07.980460 32584 server_base.cc:1034] running on GCE node
I20260811 19:03:07.981467 32584 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:03:07.983656 32584 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:03:07.984998 32584 hybrid_clock.cc:648] HybridClock initialized: now 1786474987984921 us; error 46 us; skew 500 ppm
I20260811 19:03:07.985495 32584 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:03:07 dist-test-slave-h6n0 krb5kdc[32558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474987, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:03:07.989773 32584 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 19:03:07.991751 32584 webserver.cc:460] Webserver started at http://127.0.0.1:36697/ using document root <none> and password file <none>
I20260811 19:03:07.992568 32584 fs_manager.cc:359] Metadata directory not provided
I20260811 19:03:07.992748 32584 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:03:07.993117 32584 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:03:07.996598 32584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474615205622-30146-0/minicluster-data/master-0/data/instance:
uuid: "ce106e61f0c04d1db1c93784d7305945"
format_stamp: "Formatted at 2026-08-11 19:03:07 on dist-test-slave-h6n0"
server_key: "d7019f0513be8d69842053b3356cbc82"
server_key_iv: "3260943351d0d5b84f5186a095a7fd2d"
server_key_version: "encryptionkey@0"
I20260811 19:03:07.997504 32584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474615205622-30146-0/minicluster-data/master-0/wal/instance:
uuid: "ce106e61f0c04d1db1c93784d7305945"
format_stamp: "Formatted at 2026-08-11 19:03:07 on dist-test-slave-h6n0"
server_key: "d7019f0513be8d69842053b3356cbc82"
server_key_iv: "3260943351d0d5b84f5186a095a7fd2d"
server_key_version: "encryptionkey@0"
I20260811 19:03:08.003762 32584 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260811 19:03:08.009243 32612 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:03:08.010825 32584 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 19:03:08.011015 32584 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474615205622-30146-0/minicluster-data/master-0/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474615205622-30146-0/minicluster-data/master-0/wal
uuid: "ce106e61f0c04d1db1c93784d7305945"
format_stamp: "Formatted at 2026-08-11 19:03:07 on dist-test-slave-h6n0"
server_key: "d7019f0513be8d69842053b3356cbc82"
server_key_iv: "3260943351d0d5b84f5186a095a7fd2d"
server_key_version: "encryptionkey@0"
I20260811 19:03:08.011229 32584 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474615205622-30146-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474615205622-30146-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474615205622-30146-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:03:08.017704 32584 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:03:08.022778 32584 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:03:08.023154 32584 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:03:08.046645 32584 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38845
I20260811 19:03:08.046763 32699 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38845 every 8 connection(s)
I20260811 19:03:08.048312 32584 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
I20260811 19:03:08.050630 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 32584
I20260811 19:03:08.050834 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474615205622-30146-0/minicluster-data/master-0/wal/instance
I20260811 19:03:08.051441 30146 external_mini_cluster.cc:1428] Setting key fd2bb52f3994a743ae0a79991f4696a8
I20260811 19:03:08.053198 32701 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:03:08.060757 32707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:08.053512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55709 (local address 127.0.0.1:38845)
0811 19:03:08.054420 (+ 908us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:08.054433 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:08.054476 (+ 43us) server_negotiation.cc:407] Connection header received
0811 19:03:08.056489 (+ 2013us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:08.056510 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:08.056863 (+ 353us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:08.057219 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:08.058434 (+ 1215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:03:08.058440 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:03:08.058453 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:03:08.058474 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:03:08.058634 (+ 160us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:03:08.058661 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:03:08.058857 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:08.058973 (+ 116us) server_negotiation.cc:299] Negotiation successful
0811 19:03:08.059199 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":659,"thread_start_us":425,"threads_started":1}
I20260811 19:03:08.066493 30146 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 19:03:08.068702 32701 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce106e61f0c04d1db1c93784d7305945: Bootstrap starting.
I20260811 19:03:08.072772 32701 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ce106e61f0c04d1db1c93784d7305945: Neither blocks nor log segments found. Creating new log.
I20260811 19:03:08.074299 32701 log.cc:824] T 00000000000000000000000000000000 P ce106e61f0c04d1db1c93784d7305945: Log is configured to *not* fsync() on all Append() calls
I20260811 19:03:08.074770 32707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:08.071285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47775 (local address 127.0.0.1:38845)
0811 19:03:08.071522 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:08.071528 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:08.071928 (+ 400us) server_negotiation.cc:407] Connection header received
0811 19:03:08.072116 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:08.072119 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:08.072170 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:08.072251 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:08.073831 (+ 1580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:03:08.073837 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:03:08.073841 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:03:08.073876 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:03:08.073954 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:03:08.073975 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:03:08.074133 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:08.074366 (+ 233us) server_negotiation.cc:299] Negotiation successful
0811 19:03:08.074478 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 19:03:08.077577 32701 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce106e61f0c04d1db1c93784d7305945: No bootstrap required, opened a new log
I20260811 19:03:08.085204 32701 raft_consensus.cc:348] T 00000000000000000000000000000000 P ce106e61f0c04d1db1c93784d7305945 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce106e61f0c04d1db1c93784d7305945" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38845 } }
I20260811 19:03:08.085631 32701 raft_consensus.cc:374] T 00000000000000000000000000000000 P ce106e61f0c04d1db1c93784d7305945 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:03:08.085737 32701 raft_consensus.cc:729] T 00000000000000000000000000000000 P ce106e61f0c04d1db1c93784d7305945 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce106e61f0c04d1db1c93784d7305945, State: Initialized, Role: FOLLOWER
I20260811 19:03:08.086338 32701 consensus_queue.cc:260] T 00000000000000000000000000000000 P ce106e61f0c04d1db1c93784d7305945 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce106e61f0c04d1db1c93784d7305945" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38845 } }
I20260811 19:03:08.086639 32701 raft_consensus.cc:388] T 00000000000000000000000000000000 P ce106e61f0c04d1db1c93784d7305945 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:03:08.086804 32701 raft_consensus.cc:482] T 00000000000000000000000000000000 P ce106e61f0c04d1db1c93784d7305945 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:03:08.087023 32701 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ce106e61f0c04d1db1c93784d7305945 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:03:08.089375 32701 raft_consensus.cc:504] T 00000000000000000000000000000000 P ce106e61f0c04d1db1c93784d7305945 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce106e61f0c04d1db1c93784d7305945" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38845 } }
I20260811 19:03:08.089938 32701 leader_election.cc:302] T 00000000000000000000000000000000 P ce106e61f0c04d1db1c93784d7305945 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ce106e61f0c04d1db1c93784d7305945; no voters:
I20260811 19:03:08.090626 32701 leader_election.cc:288] T 00000000000000000000000000000000 P ce106e61f0c04d1db1c93784d7305945 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:03:08.090824 32725 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ce106e61f0c04d1db1c93784d7305945 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:03:08.091581 32725 raft_consensus.cc:686] T 00000000000000000000000000000000 P ce106e61f0c04d1db1c93784d7305945 [term 1 LEADER]: Becoming Leader. State: Replica: ce106e61f0c04d1db1c93784d7305945, State: Running, Role: LEADER
I20260811 19:03:08.092217 32725 consensus_queue.cc:237] T 00000000000000000000000000000000 P ce106e61f0c04d1db1c93784d7305945 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce106e61f0c04d1db1c93784d7305945" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38845 } }
I20260811 19:03:08.092773 32701 sys_catalog.cc:564] T 00000000000000000000000000000000 P ce106e61f0c04d1db1c93784d7305945 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:03:08.095281 32729 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce106e61f0c04d1db1c93784d7305945 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ce106e61f0c04d1db1c93784d7305945" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce106e61f0c04d1db1c93784d7305945" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38845 } } }
I20260811 19:03:08.095708 32729 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce106e61f0c04d1db1c93784d7305945 [sys.catalog]: This master's current role is: LEADER
I20260811 19:03:08.096848 32744 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:03:08.096688 32731 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce106e61f0c04d1db1c93784d7305945 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce106e61f0c04d1db1c93784d7305945. Latest consensus state: current_term: 1 leader_uuid: "ce106e61f0c04d1db1c93784d7305945" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce106e61f0c04d1db1c93784d7305945" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38845 } } }
I20260811 19:03:08.099366 32731 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce106e61f0c04d1db1c93784d7305945 [sys.catalog]: This master's current role is: LEADER
I20260811 19:03:08.103566 32744 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:03:08.115696 32744 catalog_manager.cc:1347] Generated new cluster ID: 519c0e6c7c834aac9f3e43c6e6d5a74b
I20260811 19:03:08.115841 32744 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:03:08.129735 32744 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:03:08.130281 32744 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:03:08.159629 32744 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ce106e61f0c04d1db1c93784d7305945: Generated new TSK 0
I20260811 19:03:08.160856 32744 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 19:03:08.175988 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 32584
W20260811 19:03:08.177879 32693 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:55709
W20260811 19:03:08.179072 30146 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 32584: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 19:03:08.179272 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 32584
2026-08-11T19:03:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (2386 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474615205622-30146-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:03:08 dist-test-slave-h6n0 krb5kdc[32766](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:03:08 dist-test-slave-h6n0 krb5kdc[32766](info): set up 2 sockets
Aug 11 19:03:08 dist-test-slave-h6n0 krb5kdc[32766](info): commencing operation
krb5kdc: starting...
W20260811 19:03:09.702395 30146 mini_kdc.cc:121] Time spent starting KDC: real 1.470s user 0.001s sys 0.052s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474615205622-30146-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474615205622-30146-0/krb5kdc/test-user.keytab.
Aug 11 19:03:09 dist-test-slave-h6n0 krb5kdc[32766](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474989, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:03:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:03:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474615205622-30146-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474615205622-30146-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 19:03:10.073136 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474615205622-30146-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474615205622-30146-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474615205622-30146-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35263
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474615205622-30146-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35263
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:03:10.459901 316 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:03:10.460220 316 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:03:10.460394 316 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:03:10.467883 316 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:03:10.468015 316 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:03:10.468106 316 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:03:10.468225 316 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:03:10.468304 316 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:03:10.478520 316 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474615205622-30146-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474615205622-30146-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35263
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474615205622-30146-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35263
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474615205622-30146-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:03:10.479902 316 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:03:10.481345 316 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:03:10.493919 327 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:03:10.494045 329 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:03:10.494024 331 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:03:10.500519 316 server_base.cc:1034] running on GCE node
I20260811 19:03:10.501832 316 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:03:10.504331 316 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:03:10.505679 316 hybrid_clock.cc:648] HybridClock initialized: now 1786474990505603 us; error 59 us; skew 500 ppm
I20260811 19:03:10.506376 316 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:03:10 dist-test-slave-h6n0 krb5kdc[32766](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474990, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:03:10.510407 316 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 19:03:10.512817 316 webserver.cc:460] Webserver started at http://127.0.0.1:36939/ using document root <none> and password file <none>
I20260811 19:03:10.513674 316 fs_manager.cc:359] Metadata directory not provided
I20260811 19:03:10.513854 316 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:03:10.514204 316 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:03:10.517675 316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474615205622-30146-0/minicluster-data/master-0/data/instance:
uuid: "29b3330e597c416491febcd2e23e6412"
format_stamp: "Formatted at 2026-08-11 19:03:10 on dist-test-slave-h6n0"
server_key: "1898b63f912bd31582b53f3358d69439"
server_key_iv: "289518ee478dfa2d3c21c9e8772e0daf"
server_key_version: "encryptionkey@0"
I20260811 19:03:10.518589 316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474615205622-30146-0/minicluster-data/master-0/wal/instance:
uuid: "29b3330e597c416491febcd2e23e6412"
format_stamp: "Formatted at 2026-08-11 19:03:10 on dist-test-slave-h6n0"
server_key: "1898b63f912bd31582b53f3358d69439"
server_key_iv: "289518ee478dfa2d3c21c9e8772e0daf"
server_key_version: "encryptionkey@0"
I20260811 19:03:10.524464 316 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260811 19:03:10.529107 341 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:03:10.530985 316 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260811 19:03:10.531210 316 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474615205622-30146-0/minicluster-data/master-0/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474615205622-30146-0/minicluster-data/master-0/wal
uuid: "29b3330e597c416491febcd2e23e6412"
format_stamp: "Formatted at 2026-08-11 19:03:10 on dist-test-slave-h6n0"
server_key: "1898b63f912bd31582b53f3358d69439"
server_key_iv: "289518ee478dfa2d3c21c9e8772e0daf"
server_key_version: "encryptionkey@0"
I20260811 19:03:10.531446 316 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474615205622-30146-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474615205622-30146-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474615205622-30146-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:03:10.557169 316 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:03:10.563750 316 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:03:10.564126 316 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:03:10.589715 316 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35263
I20260811 19:03:10.589746 435 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35263 every 8 connection(s)
I20260811 19:03:10.591313 316 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
I20260811 19:03:10.593915 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 316
I20260811 19:03:10.594120 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474615205622-30146-0/minicluster-data/master-0/wal/instance
I20260811 19:03:10.594769 30146 external_mini_cluster.cc:1428] Setting key 32b29c15bb01f93fa89f151972fcbe13
I20260811 19:03:10.596164 438 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 19:03:10 dist-test-slave-h6n0 krb5kdc[32766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474989, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 19:03:10.610803 438 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29b3330e597c416491febcd2e23e6412: Bootstrap starting.
I20260811 19:03:10.616303 438 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 29b3330e597c416491febcd2e23e6412: Neither blocks nor log segments found. Creating new log.
I20260811 19:03:10.617559 438 log.cc:824] T 00000000000000000000000000000000 P 29b3330e597c416491febcd2e23e6412: Log is configured to *not* fsync() on all Append() calls
I20260811 19:03:10.622366 438 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29b3330e597c416491febcd2e23e6412: No bootstrap required, opened a new log
I20260811 19:03:10.626750 447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:10.597101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36623 (local address 127.0.0.1:35263)
0811 19:03:10.597947 (+ 846us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:10.597960 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:10.598003 (+ 43us) server_negotiation.cc:407] Connection header received
0811 19:03:10.599814 (+ 1811us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:10.599836 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:10.600232 (+ 396us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:10.600686 (+ 454us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:10.602324 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:10.603959 (+ 1635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:10.605328 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:10.606052 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:10.606275 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:10.611088 (+ 4813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:03:10.611097 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:03:10.611117 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:03:10.611160 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:03:10.621817 (+ 10657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:10.622830 (+ 1013us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:10.622838 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:10.622845 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:10.623050 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:10.623494 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:10.623499 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:10.623502 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:10.624107 (+ 605us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:03:10.624445 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:10.625023 (+ 578us) server_negotiation.cc:299] Negotiation successful
0811 19:03:10.625482 (+ 459us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":629,"thread_start_us":388,"threads_started":1}
I20260811 19:03:10.632786 30146 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 19:03:10.633733 438 raft_consensus.cc:348] T 00000000000000000000000000000000 P 29b3330e597c416491febcd2e23e6412 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29b3330e597c416491febcd2e23e6412" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35263 } }
I20260811 19:03:10.634162 438 raft_consensus.cc:374] T 00000000000000000000000000000000 P 29b3330e597c416491febcd2e23e6412 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:03:10.634279 438 raft_consensus.cc:729] T 00000000000000000000000000000000 P 29b3330e597c416491febcd2e23e6412 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29b3330e597c416491febcd2e23e6412, State: Initialized, Role: FOLLOWER
I20260811 19:03:10.634956 438 consensus_queue.cc:260] T 00000000000000000000000000000000 P 29b3330e597c416491febcd2e23e6412 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29b3330e597c416491febcd2e23e6412" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35263 } }
I20260811 19:03:10.635229 438 raft_consensus.cc:388] T 00000000000000000000000000000000 P 29b3330e597c416491febcd2e23e6412 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:03:10.635428 438 raft_consensus.cc:482] T 00000000000000000000000000000000 P 29b3330e597c416491febcd2e23e6412 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:03:10.635646 438 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 29b3330e597c416491febcd2e23e6412 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:03:10.639041 438 raft_consensus.cc:504] T 00000000000000000000000000000000 P 29b3330e597c416491febcd2e23e6412 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29b3330e597c416491febcd2e23e6412" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35263 } }
I20260811 19:03:10.639678 438 leader_election.cc:302] T 00000000000000000000000000000000 P 29b3330e597c416491febcd2e23e6412 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 29b3330e597c416491febcd2e23e6412; no voters:
I20260811 19:03:10.640326 438 leader_election.cc:288] T 00000000000000000000000000000000 P 29b3330e597c416491febcd2e23e6412 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:03:10.640488 461 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 29b3330e597c416491febcd2e23e6412 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:03:10.641462 461 raft_consensus.cc:686] T 00000000000000000000000000000000 P 29b3330e597c416491febcd2e23e6412 [term 1 LEADER]: Becoming Leader. State: Replica: 29b3330e597c416491febcd2e23e6412, State: Running, Role: LEADER
I20260811 19:03:10.642560 461 consensus_queue.cc:237] T 00000000000000000000000000000000 P 29b3330e597c416491febcd2e23e6412 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29b3330e597c416491febcd2e23e6412" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35263 } }
I20260811 19:03:10.644270 438 sys_catalog.cc:564] T 00000000000000000000000000000000 P 29b3330e597c416491febcd2e23e6412 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:03:10.645545 464 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29b3330e597c416491febcd2e23e6412 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "29b3330e597c416491febcd2e23e6412" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29b3330e597c416491febcd2e23e6412" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35263 } } }
I20260811 19:03:10.645969 464 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29b3330e597c416491febcd2e23e6412 [sys.catalog]: This master's current role is: LEADER
I20260811 19:03:10.646512 463 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29b3330e597c416491febcd2e23e6412 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 29b3330e597c416491febcd2e23e6412. Latest consensus state: current_term: 1 leader_uuid: "29b3330e597c416491febcd2e23e6412" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29b3330e597c416491febcd2e23e6412" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35263 } } }
I20260811 19:03:10.647117 463 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29b3330e597c416491febcd2e23e6412 [sys.catalog]: This master's current role is: LEADER
I20260811 19:03:10.653518 468 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:03:10.660800 447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:10.638478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37549 (local address 127.0.0.1:35263)
0811 19:03:10.638701 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:10.638706 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:10.638732 (+ 26us) server_negotiation.cc:407] Connection header received
0811 19:03:10.638870 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:10.638878 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:10.638973 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:10.639098 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:10.641924 (+ 2826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:10.643315 (+ 1391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:10.645474 (+ 2159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:10.646374 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:10.646504 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:10.651215 (+ 4711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:03:10.651225 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:03:10.651232 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:03:10.651291 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:03:10.655775 (+ 4484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:10.656808 (+ 1033us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:10.656813 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:10.656817 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:10.657063 (+ 246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:10.657727 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:10.657732 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:10.657735 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:10.658227 (+ 492us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:03:10.658524 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:10.659195 (+ 671us) server_negotiation.cc:299] Negotiation successful
0811 19:03:10.659474 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":89}
I20260811 19:03:10.663220 468 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:03:10.680757 468 catalog_manager.cc:1347] Generated new cluster ID: 4210d9a37e4e49ce88528fe49ad237be
I20260811 19:03:10.681000 468 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:03:10.721992 468 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:03:10.724277 468 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:03:10.742201 468 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 29b3330e597c416491febcd2e23e6412: Generated new TSK 0
I20260811 19:03:10.743690 468 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 19:03:10.766491 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 316
I20260811 19:03:10.767684 428 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:03:10.868857 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 316
2026-08-11T19:03:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2688 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474615205622-30146-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:03:11 dist-test-slave-h6n0 krb5kdc[497](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:03:11 dist-test-slave-h6n0 krb5kdc[497](info): set up 2 sockets
Aug 11 19:03:11 dist-test-slave-h6n0 krb5kdc[497](info): commencing operation
krb5kdc: starting...
W20260811 19:03:12.328622 30146 mini_kdc.cc:121] Time spent starting KDC: real 1.409s user 0.000s sys 0.049s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474615205622-30146-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474615205622-30146-0/krb5kdc/test-user.keytab.
Aug 11 19:03:12 dist-test-slave-h6n0 krb5kdc[497](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474992, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:03:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:03:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474615205622-30146-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474615205622-30146-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 19:03:12.739997 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474615205622-30146-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474615205622-30146-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474615205622-30146-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45395
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474615205622-30146-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36217
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45395
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:03:13.153167 525 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:03:13.153635 525 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:03:13.153820 525 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:03:13.162689 525 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:03:13.162889 525 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:03:13.163000 525 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:03:13.163118 525 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:03:13.163205 525 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:03:13.176543 525 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36217
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474615205622-30146-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474615205622-30146-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45395
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474615205622-30146-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45395
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474615205622-30146-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:03:13.178117 525 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:03:13.179904 525 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:03:13.194576 542 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:03:13.194666 534 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:03:13.194679 531 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:03:13.196079 525 server_base.cc:1034] running on GCE node
I20260811 19:03:13.197928 525 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:03:13.200122 525 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:03:13.201509 525 hybrid_clock.cc:648] HybridClock initialized: now 1786474993201442 us; error 68 us; skew 500 ppm
I20260811 19:03:13.201951 525 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:03:13 dist-test-slave-h6n0 krb5kdc[497](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474993, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:03:13.207149 525 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 19:03:13.209224 525 webserver.cc:460] Webserver started at http://127.0.0.1:32983/ using document root <none> and password file <none>
I20260811 19:03:13.209977 525 fs_manager.cc:359] Metadata directory not provided
I20260811 19:03:13.210144 525 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:03:13.210487 525 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:03:13.213749 525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474615205622-30146-0/minicluster-data/master-0/data/instance:
uuid: "3242cadf6f1e4b86a6516b3ce2d34f34"
format_stamp: "Formatted at 2026-08-11 19:03:13 on dist-test-slave-h6n0"
server_key: "343bc5fd3d298da7ba525e740b2518cd"
server_key_iv: "0575be04c5336475963dc55841cfa4d3"
server_key_version: "encryptionkey@0"
I20260811 19:03:13.214699 525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474615205622-30146-0/minicluster-data/master-0/wal/instance:
uuid: "3242cadf6f1e4b86a6516b3ce2d34f34"
format_stamp: "Formatted at 2026-08-11 19:03:13 on dist-test-slave-h6n0"
server_key: "343bc5fd3d298da7ba525e740b2518cd"
server_key_iv: "0575be04c5336475963dc55841cfa4d3"
server_key_version: "encryptionkey@0"
I20260811 19:03:13.224136 525 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.003s sys 0.005s
I20260811 19:03:13.229379 553 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:03:13.231745 525 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.001s
I20260811 19:03:13.232038 525 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474615205622-30146-0/minicluster-data/master-0/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474615205622-30146-0/minicluster-data/master-0/wal
uuid: "3242cadf6f1e4b86a6516b3ce2d34f34"
format_stamp: "Formatted at 2026-08-11 19:03:13 on dist-test-slave-h6n0"
server_key: "343bc5fd3d298da7ba525e740b2518cd"
server_key_iv: "0575be04c5336475963dc55841cfa4d3"
server_key_version: "encryptionkey@0"
I20260811 19:03:13.232363 525 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474615205622-30146-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474615205622-30146-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474615205622-30146-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:03:13.263468 525 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:03:13.270272 525 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:03:13.270759 525 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:03:13.295732 525 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45395
I20260811 19:03:13.295778 642 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45395 every 8 connection(s)
I20260811 19:03:13.297363 525 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
I20260811 19:03:13.300433 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 525
I20260811 19:03:13.300729 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474615205622-30146-0/minicluster-data/master-0/wal/instance
I20260811 19:03:13.301666 30146 external_mini_cluster.cc:1428] Setting key 1e11efd71703a78d9078745e210f32e7
I20260811 19:03:13.302724 644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:03:13.320602 644 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3242cadf6f1e4b86a6516b3ce2d34f34: Bootstrap starting.
I20260811 19:03:13.321170 651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:13.304401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45815 (local address 127.0.0.1:45395)
0811 19:03:13.305477 (+ 1076us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:13.305497 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:13.305561 (+ 64us) server_negotiation.cc:407] Connection header received
0811 19:03:13.309095 (+ 3534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:13.309138 (+ 43us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:13.309996 (+ 858us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:13.310570 (+ 574us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:13.311213 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:13.313341 (+ 2128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:13.315428 (+ 2087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:13.316583 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:13.316821 (+ 238us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:13.318180 (+ 1359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:03:13.318193 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:03:13.318214 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:03:13.319012 (+ 798us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:03:13.319130 (+ 118us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:03:13.319167 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:03:13.319287 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:13.319534 (+ 247us) server_negotiation.cc:299] Negotiation successful
0811 19:03:13.319830 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":753,"thread_start_us":453,"threads_started":1}
I20260811 19:03:13.324110 644 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3242cadf6f1e4b86a6516b3ce2d34f34: Neither blocks nor log segments found. Creating new log.
I20260811 19:03:13.325152 30146 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 19:03:13.325361 644 log.cc:824] T 00000000000000000000000000000000 P 3242cadf6f1e4b86a6516b3ce2d34f34: Log is configured to *not* fsync() on all Append() calls
I20260811 19:03:13.328622 644 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3242cadf6f1e4b86a6516b3ce2d34f34: No bootstrap required, opened a new log
I20260811 19:03:13.339367 651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:13.330302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57691 (local address 127.0.0.1:45395)
0811 19:03:13.330564 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:13.330573 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:13.331178 (+ 605us) server_negotiation.cc:407] Connection header received
0811 19:03:13.331436 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:13.331440 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:13.331512 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:13.331607 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:13.332375 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:13.333219 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:13.335408 (+ 2189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:13.336377 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:13.336576 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:13.338048 (+ 1472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:03:13.338056 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:03:13.338060 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:03:13.338124 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:03:13.338197 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:03:13.338224 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:03:13.338361 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:13.338683 (+ 322us) server_negotiation.cc:299] Negotiation successful
0811 19:03:13.338856 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 19:03:13.340382 644 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3242cadf6f1e4b86a6516b3ce2d34f34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3242cadf6f1e4b86a6516b3ce2d34f34" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45395 } }
I20260811 19:03:13.340863 644 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3242cadf6f1e4b86a6516b3ce2d34f34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:03:13.341017 644 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3242cadf6f1e4b86a6516b3ce2d34f34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3242cadf6f1e4b86a6516b3ce2d34f34, State: Initialized, Role: FOLLOWER
I20260811 19:03:13.341744 644 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3242cadf6f1e4b86a6516b3ce2d34f34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3242cadf6f1e4b86a6516b3ce2d34f34" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45395 } }
I20260811 19:03:13.342024 644 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3242cadf6f1e4b86a6516b3ce2d34f34 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:03:13.342196 644 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3242cadf6f1e4b86a6516b3ce2d34f34 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:03:13.342384 644 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3242cadf6f1e4b86a6516b3ce2d34f34 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:03:13.345731 644 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3242cadf6f1e4b86a6516b3ce2d34f34 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3242cadf6f1e4b86a6516b3ce2d34f34" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45395 } }
I20260811 19:03:13.346748 644 leader_election.cc:302] T 00000000000000000000000000000000 P 3242cadf6f1e4b86a6516b3ce2d34f34 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3242cadf6f1e4b86a6516b3ce2d34f34; no voters:
I20260811 19:03:13.347604 644 leader_election.cc:288] T 00000000000000000000000000000000 P 3242cadf6f1e4b86a6516b3ce2d34f34 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:03:13.347811 660 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3242cadf6f1e4b86a6516b3ce2d34f34 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:03:13.348802 660 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3242cadf6f1e4b86a6516b3ce2d34f34 [term 1 LEADER]: Becoming Leader. State: Replica: 3242cadf6f1e4b86a6516b3ce2d34f34, State: Running, Role: LEADER
I20260811 19:03:13.349526 660 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3242cadf6f1e4b86a6516b3ce2d34f34 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3242cadf6f1e4b86a6516b3ce2d34f34" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45395 } }
I20260811 19:03:13.350451 644 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3242cadf6f1e4b86a6516b3ce2d34f34 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:03:13.353071 666 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3242cadf6f1e4b86a6516b3ce2d34f34 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3242cadf6f1e4b86a6516b3ce2d34f34. Latest consensus state: current_term: 1 leader_uuid: "3242cadf6f1e4b86a6516b3ce2d34f34" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3242cadf6f1e4b86a6516b3ce2d34f34" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45395 } } }
I20260811 19:03:13.353698 666 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3242cadf6f1e4b86a6516b3ce2d34f34 [sys.catalog]: This master's current role is: LEADER
I20260811 19:03:13.353772 661 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3242cadf6f1e4b86a6516b3ce2d34f34 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3242cadf6f1e4b86a6516b3ce2d34f34" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3242cadf6f1e4b86a6516b3ce2d34f34" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45395 } } }
I20260811 19:03:13.354255 661 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3242cadf6f1e4b86a6516b3ce2d34f34 [sys.catalog]: This master's current role is: LEADER
I20260811 19:03:13.358148 669 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:03:13.365658 669 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:03:13.376408 669 catalog_manager.cc:1347] Generated new cluster ID: 2047bd4b549f47b7868ee44180994756
I20260811 19:03:13.376559 669 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:03:13.396665 669 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:03:13.399505 669 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:03:13.408262 669 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3242cadf6f1e4b86a6516b3ce2d34f34: Generated new TSK 0
I20260811 19:03:13.409412 669 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 19:03:13.416322 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 525
W20260811 19:03:13.417935 630 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:45815
W20260811 19:03:13.419361 30146 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 525: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 19:03:13.419763 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 525
2026-08-11T19:03:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (2559 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474615205622-30146-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:03:13 dist-test-slave-h6n0 krb5kdc[698](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:03:13 dist-test-slave-h6n0 krb5kdc[698](info): set up 2 sockets
Aug 11 19:03:13 dist-test-slave-h6n0 krb5kdc[698](info): commencing operation
krb5kdc: starting...
W20260811 19:03:14.957620 30146 mini_kdc.cc:121] Time spent starting KDC: real 1.479s user 0.003s sys 0.058s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474615205622-30146-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474615205622-30146-0/krb5kdc/test-user.keytab.
Aug 11 19:03:15 dist-test-slave-h6n0 krb5kdc[698](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474995, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:03:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:03:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474615205622-30146-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474615205622-30146-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 19:03:15.329715 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474615205622-30146-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474615205622-30146-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474615205622-30146-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36079
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474615205622-30146-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36079
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:03:15.725039 725 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:03:15.725443 725 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:03:15.725706 725 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:03:15.734467 725 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:03:15.734671 725 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:03:15.734755 725 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:03:15.734860 725 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:03:15.734925 725 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:03:15.747011 725 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474615205622-30146-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474615205622-30146-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36079
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474615205622-30146-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36079
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474615205622-30146-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:03:15.748868 725 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:03:15.751139 725 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:03:15.766400 734 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:03:15.766362 735 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:03:15.766367 737 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:03:15.767668 725 server_base.cc:1034] running on GCE node
I20260811 19:03:15.768631 725 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:03:15.770149 725 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:03:15.771432 725 hybrid_clock.cc:648] HybridClock initialized: now 1786474995771339 us; error 86 us; skew 500 ppm
I20260811 19:03:15.771806 725 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:03:15 dist-test-slave-h6n0 krb5kdc[698](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474995, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:03:15.783368 725 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 19:03:15.785409 725 webserver.cc:460] Webserver started at http://127.0.0.1:37575/ using document root <none> and password file <none>
I20260811 19:03:15.786208 725 fs_manager.cc:359] Metadata directory not provided
I20260811 19:03:15.786373 725 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:03:15.786811 725 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:03:15.790251 725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474615205622-30146-0/minicluster-data/master-0/data/instance:
uuid: "4bd975f756ec4993b82e5d4c0deb9227"
format_stamp: "Formatted at 2026-08-11 19:03:15 on dist-test-slave-h6n0"
server_key: "6eb0d95092c0a820d202a48ef97fe279"
server_key_iv: "7959879950bd5031614b3ddd93be7cfa"
server_key_version: "encryptionkey@0"
I20260811 19:03:15.791247 725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474615205622-30146-0/minicluster-data/master-0/wal/instance:
uuid: "4bd975f756ec4993b82e5d4c0deb9227"
format_stamp: "Formatted at 2026-08-11 19:03:15 on dist-test-slave-h6n0"
server_key: "6eb0d95092c0a820d202a48ef97fe279"
server_key_iv: "7959879950bd5031614b3ddd93be7cfa"
server_key_version: "encryptionkey@0"
I20260811 19:03:15.797576 725 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 19:03:15.801671 753 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:03:15.803225 725 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 19:03:15.803422 725 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474615205622-30146-0/minicluster-data/master-0/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474615205622-30146-0/minicluster-data/master-0/wal
uuid: "4bd975f756ec4993b82e5d4c0deb9227"
format_stamp: "Formatted at 2026-08-11 19:03:15 on dist-test-slave-h6n0"
server_key: "6eb0d95092c0a820d202a48ef97fe279"
server_key_iv: "7959879950bd5031614b3ddd93be7cfa"
server_key_version: "encryptionkey@0"
I20260811 19:03:15.803655 725 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474615205622-30146-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474615205622-30146-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474615205622-30146-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:03:15.810042 725 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:03:15.815678 725 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:03:15.816030 725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:03:15.839911 725 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36079
I20260811 19:03:15.839927 848 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36079 every 8 connection(s)
I20260811 19:03:15.841818 725 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
I20260811 19:03:15.846577 849 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:03:15.848817 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 725
I20260811 19:03:15.849056 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474615205622-30146-0/minicluster-data/master-0/wal/instance
I20260811 19:03:15.849550 30146 external_mini_cluster.cc:1428] Setting key 449af37ab8ea820af8288ea4d355c853
I20260811 19:03:15.858898 855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:15.851545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46573 (local address 127.0.0.1:36079)
0811 19:03:15.852487 (+ 942us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:15.852510 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:15.852570 (+ 60us) server_negotiation.cc:407] Connection header received
0811 19:03:15.854466 (+ 1896us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:15.854497 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:15.854994 (+ 497us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:15.855442 (+ 448us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:15.856560 (+ 1118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:03:15.856570 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:03:15.856588 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:03:15.856610 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:03:15.856692 (+ 82us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:03:15.856728 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:03:15.856994 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:15.857316 (+ 322us) server_negotiation.cc:299] Negotiation successful
0811 19:03:15.857517 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":675,"thread_start_us":413,"threads_started":1}
I20260811 19:03:15.862921 849 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4bd975f756ec4993b82e5d4c0deb9227: Bootstrap starting.
I20260811 19:03:15.864095 30146 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 19:03:15.866225 849 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4bd975f756ec4993b82e5d4c0deb9227: Neither blocks nor log segments found. Creating new log.
I20260811 19:03:15.867496 849 log.cc:824] T 00000000000000000000000000000000 P 4bd975f756ec4993b82e5d4c0deb9227: Log is configured to *not* fsync() on all Append() calls
I20260811 19:03:15.871002 849 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4bd975f756ec4993b82e5d4c0deb9227: No bootstrap required, opened a new log
I20260811 19:03:15.871788 855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:15.868816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39405 (local address 127.0.0.1:36079)
0811 19:03:15.869051 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:15.869057 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:15.869420 (+ 363us) server_negotiation.cc:407] Connection header received
0811 19:03:15.869575 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:15.869578 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:15.869625 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:15.869707 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:15.870886 (+ 1179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:03:15.870894 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:03:15.870898 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:03:15.870927 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:03:15.871006 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:03:15.871030 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:03:15.871379 (+ 349us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:15.871455 (+ 76us) server_negotiation.cc:299] Negotiation successful
0811 19:03:15.871522 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 19:03:15.879699 849 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4bd975f756ec4993b82e5d4c0deb9227 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bd975f756ec4993b82e5d4c0deb9227" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36079 } }
I20260811 19:03:15.880223 849 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4bd975f756ec4993b82e5d4c0deb9227 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:03:15.880366 849 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4bd975f756ec4993b82e5d4c0deb9227 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bd975f756ec4993b82e5d4c0deb9227, State: Initialized, Role: FOLLOWER
I20260811 19:03:15.880940 849 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4bd975f756ec4993b82e5d4c0deb9227 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bd975f756ec4993b82e5d4c0deb9227" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36079 } }
I20260811 19:03:15.881291 849 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4bd975f756ec4993b82e5d4c0deb9227 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:03:15.881461 849 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4bd975f756ec4993b82e5d4c0deb9227 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:03:15.881640 849 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4bd975f756ec4993b82e5d4c0deb9227 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:03:15.885866 849 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4bd975f756ec4993b82e5d4c0deb9227 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bd975f756ec4993b82e5d4c0deb9227" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36079 } }
I20260811 19:03:15.886631 849 leader_election.cc:302] T 00000000000000000000000000000000 P 4bd975f756ec4993b82e5d4c0deb9227 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4bd975f756ec4993b82e5d4c0deb9227; no voters:
I20260811 19:03:15.887564 849 leader_election.cc:288] T 00000000000000000000000000000000 P 4bd975f756ec4993b82e5d4c0deb9227 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:03:15.887881 865 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4bd975f756ec4993b82e5d4c0deb9227 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:03:15.891963 865 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4bd975f756ec4993b82e5d4c0deb9227 [term 1 LEADER]: Becoming Leader. State: Replica: 4bd975f756ec4993b82e5d4c0deb9227, State: Running, Role: LEADER
I20260811 19:03:15.892699 865 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4bd975f756ec4993b82e5d4c0deb9227 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bd975f756ec4993b82e5d4c0deb9227" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36079 } }
I20260811 19:03:15.893463 849 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4bd975f756ec4993b82e5d4c0deb9227 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:03:15.900853 866 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4bd975f756ec4993b82e5d4c0deb9227 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4bd975f756ec4993b82e5d4c0deb9227" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bd975f756ec4993b82e5d4c0deb9227" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36079 } } }
I20260811 19:03:15.899223 868 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4bd975f756ec4993b82e5d4c0deb9227 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4bd975f756ec4993b82e5d4c0deb9227. Latest consensus state: current_term: 1 leader_uuid: "4bd975f756ec4993b82e5d4c0deb9227" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bd975f756ec4993b82e5d4c0deb9227" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36079 } } }
I20260811 19:03:15.901682 866 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4bd975f756ec4993b82e5d4c0deb9227 [sys.catalog]: This master's current role is: LEADER
I20260811 19:03:15.901808 868 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4bd975f756ec4993b82e5d4c0deb9227 [sys.catalog]: This master's current role is: LEADER
I20260811 19:03:15.902721 882 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:03:15.908703 882 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:03:15.920292 882 catalog_manager.cc:1347] Generated new cluster ID: 1653e1b606974237b2d3dd27c2835dee
I20260811 19:03:15.920445 882 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:03:15.965190 882 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:03:15.965911 882 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:03:15.975024 882 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4bd975f756ec4993b82e5d4c0deb9227: Generated new TSK 0
I20260811 19:03:15.975973 882 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 19:03:15.997483 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 725
W20260811 19:03:15.999131 840 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:46573
W20260811 19:03:16.000146 30146 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 725: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 19:03:16.000344 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 725
2026-08-11T19:03:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (2577 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474615205622-30146-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:03:16 dist-test-slave-h6n0 krb5kdc[905](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:03:16 dist-test-slave-h6n0 krb5kdc[905](info): set up 2 sockets
Aug 11 19:03:16 dist-test-slave-h6n0 krb5kdc[905](info): commencing operation
krb5kdc: starting...
W20260811 19:03:17.475587 30146 mini_kdc.cc:121] Time spent starting KDC: real 1.420s user 0.000s sys 0.051s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474615205622-30146-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474615205622-30146-0/krb5kdc/test-user.keytab.
Aug 11 19:03:17 dist-test-slave-h6n0 krb5kdc[905](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474997, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:03:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:03:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474615205622-30146-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474615205622-30146-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 19:03:17.846871 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474615205622-30146-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474615205622-30146-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474615205622-30146-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38433
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474615205622-30146-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38433
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:03:18.212337 930 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:03:18.212688 930 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:03:18.212849 930 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:03:18.221489 930 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:03:18.221648 930 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:03:18.221699 930 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:03:18.221834 930 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:03:18.221925 930 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:03:18.232884 930 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474615205622-30146-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474615205622-30146-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38433
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474615205622-30146-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38433
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474615205622-30146-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:03:18.234634 930 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:03:18.236184 930 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:03:18.253304 940 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:03:18.253278 936 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:03:18.253976 946 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:03:18.259071 930 server_base.cc:1034] running on GCE node
I20260811 19:03:18.261039 930 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:03:18.263016 930 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:03:18.264645 930 hybrid_clock.cc:648] HybridClock initialized: now 1786474998264590 us; error 51 us; skew 500 ppm
I20260811 19:03:18.265225 930 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:03:18 dist-test-slave-h6n0 krb5kdc[905](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474998, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:03:18.273022 930 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 19:03:18.277053 930 webserver.cc:460] Webserver started at http://127.0.0.1:44675/ using document root <none> and password file <none>
I20260811 19:03:18.278064 930 fs_manager.cc:359] Metadata directory not provided
I20260811 19:03:18.278231 930 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:03:18.278661 930 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:03:18.285813 930 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474615205622-30146-0/minicluster-data/master-0/data/instance:
uuid: "f39aaf0bed674d5797223fd2efe3ce2f"
format_stamp: "Formatted at 2026-08-11 19:03:18 on dist-test-slave-h6n0"
server_key: "2968987f3adde85cfc0a93c71277c1c8"
server_key_iv: "c83de6d1159fed6d28c8adeba40a1f25"
server_key_version: "encryptionkey@0"
I20260811 19:03:18.287169 930 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474615205622-30146-0/minicluster-data/master-0/wal/instance:
uuid: "f39aaf0bed674d5797223fd2efe3ce2f"
format_stamp: "Formatted at 2026-08-11 19:03:18 on dist-test-slave-h6n0"
server_key: "2968987f3adde85cfc0a93c71277c1c8"
server_key_iv: "c83de6d1159fed6d28c8adeba40a1f25"
server_key_version: "encryptionkey@0"
I20260811 19:03:18.294941 930 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.002s sys 0.004s
I20260811 19:03:18.300156 956 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:03:18.302083 930 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.001s
I20260811 19:03:18.302320 930 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474615205622-30146-0/minicluster-data/master-0/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474615205622-30146-0/minicluster-data/master-0/wal
uuid: "f39aaf0bed674d5797223fd2efe3ce2f"
format_stamp: "Formatted at 2026-08-11 19:03:18 on dist-test-slave-h6n0"
server_key: "2968987f3adde85cfc0a93c71277c1c8"
server_key_iv: "c83de6d1159fed6d28c8adeba40a1f25"
server_key_version: "encryptionkey@0"
I20260811 19:03:18.302609 930 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474615205622-30146-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474615205622-30146-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474615205622-30146-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:03:18.331720 930 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:03:18.337642 930 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:03:18.338073 930 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:03:18.367139 930 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38433
I20260811 19:03:18.367158 1047 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38433 every 8 connection(s)
I20260811 19:03:18.369302 930 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
I20260811 19:03:18.373849 1048 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:03:18.375548 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 930
I20260811 19:03:18.375787 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474615205622-30146-0/minicluster-data/master-0/wal/instance
I20260811 19:03:18.376322 30146 external_mini_cluster.cc:1428] Setting key 0342b25510f7c276d620b9ed385debe2
I20260811 19:03:18.389798 1048 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f39aaf0bed674d5797223fd2efe3ce2f: Bootstrap starting.
I20260811 19:03:18.393709 1048 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f39aaf0bed674d5797223fd2efe3ce2f: Neither blocks nor log segments found. Creating new log.
I20260811 19:03:18.394953 1048 log.cc:824] T 00000000000000000000000000000000 P f39aaf0bed674d5797223fd2efe3ce2f: Log is configured to *not* fsync() on all Append() calls
Aug 11 19:03:18 dist-test-slave-h6n0 krb5kdc[905](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474997, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 19:03:18.398661 1048 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f39aaf0bed674d5797223fd2efe3ce2f: No bootstrap required, opened a new log
I20260811 19:03:18.410460 1048 raft_consensus.cc:348] T 00000000000000000000000000000000 P f39aaf0bed674d5797223fd2efe3ce2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f39aaf0bed674d5797223fd2efe3ce2f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38433 } }
I20260811 19:03:18.411144 1048 raft_consensus.cc:374] T 00000000000000000000000000000000 P f39aaf0bed674d5797223fd2efe3ce2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:03:18.411516 1048 raft_consensus.cc:729] T 00000000000000000000000000000000 P f39aaf0bed674d5797223fd2efe3ce2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f39aaf0bed674d5797223fd2efe3ce2f, State: Initialized, Role: FOLLOWER
I20260811 19:03:18.412298 1048 consensus_queue.cc:260] T 00000000000000000000000000000000 P f39aaf0bed674d5797223fd2efe3ce2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f39aaf0bed674d5797223fd2efe3ce2f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38433 } }
I20260811 19:03:18.412573 1048 raft_consensus.cc:388] T 00000000000000000000000000000000 P f39aaf0bed674d5797223fd2efe3ce2f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:03:18.412776 1048 raft_consensus.cc:482] T 00000000000000000000000000000000 P f39aaf0bed674d5797223fd2efe3ce2f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:03:18.412969 1053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:18.379613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.52:35087 (local address 127.0.0.1:38433)
0811 19:03:18.381011 (+ 1398us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:18.381030 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:18.381088 (+ 58us) server_negotiation.cc:407] Connection header received
0811 19:03:18.383655 (+ 2567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:18.383684 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:18.384246 (+ 562us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:18.384727 (+ 481us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:18.387083 (+ 2356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:18.389542 (+ 2459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:18.391972 (+ 2430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:18.393071 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:18.393273 (+ 202us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:18.399796 (+ 6523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:03:18.399812 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:03:18.399834 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:03:18.399892 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:03:18.404522 (+ 4630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:18.406096 (+ 1574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:18.406107 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:18.406116 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:18.406360 (+ 244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:18.407853 (+ 1493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:18.407860 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:18.407863 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:18.408970 (+ 1107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:03:18.409537 (+ 567us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:18.410184 (+ 647us) server_negotiation.cc:299] Negotiation successful
0811 19:03:18.410782 (+ 598us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1098,"thread_start_us":763,"threads_started":1}
I20260811 19:03:18.412986 1048 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f39aaf0bed674d5797223fd2efe3ce2f [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:03:18.420384 1048 raft_consensus.cc:504] T 00000000000000000000000000000000 P f39aaf0bed674d5797223fd2efe3ce2f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f39aaf0bed674d5797223fd2efe3ce2f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38433 } }
I20260811 19:03:18.422155 30146 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 19:03:18.422088 1048 leader_election.cc:302] T 00000000000000000000000000000000 P f39aaf0bed674d5797223fd2efe3ce2f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f39aaf0bed674d5797223fd2efe3ce2f; no voters:
I20260811 19:03:18.423251 1048 leader_election.cc:288] T 00000000000000000000000000000000 P f39aaf0bed674d5797223fd2efe3ce2f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:03:18.423365 1056 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f39aaf0bed674d5797223fd2efe3ce2f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:03:18.428921 1056 raft_consensus.cc:686] T 00000000000000000000000000000000 P f39aaf0bed674d5797223fd2efe3ce2f [term 1 LEADER]: Becoming Leader. State: Replica: f39aaf0bed674d5797223fd2efe3ce2f, State: Running, Role: LEADER
I20260811 19:03:18.429641 1056 consensus_queue.cc:237] T 00000000000000000000000000000000 P f39aaf0bed674d5797223fd2efe3ce2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f39aaf0bed674d5797223fd2efe3ce2f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38433 } }
I20260811 19:03:18.430363 1048 sys_catalog.cc:564] T 00000000000000000000000000000000 P f39aaf0bed674d5797223fd2efe3ce2f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:03:18.432828 1057 sys_catalog.cc:455] T 00000000000000000000000000000000 P f39aaf0bed674d5797223fd2efe3ce2f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f39aaf0bed674d5797223fd2efe3ce2f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f39aaf0bed674d5797223fd2efe3ce2f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38433 } } }
I20260811 19:03:18.433229 1057 sys_catalog.cc:458] T 00000000000000000000000000000000 P f39aaf0bed674d5797223fd2efe3ce2f [sys.catalog]: This master's current role is: LEADER
I20260811 19:03:18.433899 1060 sys_catalog.cc:455] T 00000000000000000000000000000000 P f39aaf0bed674d5797223fd2efe3ce2f [sys.catalog]: SysCatalogTable state changed. Reason: New leader f39aaf0bed674d5797223fd2efe3ce2f. Latest consensus state: current_term: 1 leader_uuid: "f39aaf0bed674d5797223fd2efe3ce2f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f39aaf0bed674d5797223fd2efe3ce2f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38433 } } }
I20260811 19:03:18.434262 1060 sys_catalog.cc:458] T 00000000000000000000000000000000 P f39aaf0bed674d5797223fd2efe3ce2f [sys.catalog]: This master's current role is: LEADER
I20260811 19:03:18.434415 1076 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:03:18.456434 1076 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:03:18.465751 1053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:18.429025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.52:35317 (local address 127.0.0.1:38433)
0811 19:03:18.429185 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:18.429193 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:18.432325 (+ 3132us) server_negotiation.cc:407] Connection header received
0811 19:03:18.432583 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:18.432592 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:18.432699 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:18.432852 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:18.435385 (+ 2533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:18.436991 (+ 1606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:18.439973 (+ 2982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:18.440942 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:18.441076 (+ 134us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:18.447187 (+ 6111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:03:18.447204 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:03:18.447213 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:03:18.447277 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:03:18.460269 (+ 12992us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:18.461670 (+ 1401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:18.461678 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:18.461682 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:18.461876 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:18.462616 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:18.462622 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:18.462627 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:18.463133 (+ 506us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:03:18.463454 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:18.464952 (+ 1498us) server_negotiation.cc:299] Negotiation successful
0811 19:03:18.465264 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260811 19:03:18.474249 1076 catalog_manager.cc:1347] Generated new cluster ID: cf4c199fb6a0402eb45d14e270d7e781
I20260811 19:03:18.474462 1076 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:03:18.491243 1076 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:03:18.493278 1076 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:03:18.504842 1076 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f39aaf0bed674d5797223fd2efe3ce2f: Generated new TSK 0
I20260811 19:03:18.505800 1076 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 19:03:18.517709 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 930
I20260811 19:03:18.519515 1042 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:03:18.625561 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 930
W20260811 19:03:18.657855 903 connection.cc:535] client connection to 127.0.0.1:38433 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38433): Connection reset by peer (error 104)
2026-08-11T19:03:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2623 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474615205622-30146-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:03:18 dist-test-slave-h6n0 krb5kdc[1102](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:03:18 dist-test-slave-h6n0 krb5kdc[1102](info): set up 2 sockets
Aug 11 19:03:18 dist-test-slave-h6n0 krb5kdc[1102](info): commencing operation
krb5kdc: starting...
W20260811 19:03:20.124974 30146 mini_kdc.cc:121] Time spent starting KDC: real 1.446s user 0.000s sys 0.054s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474615205622-30146-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474615205622-30146-0/krb5kdc/test-user.keytab.
Aug 11 19:03:20 dist-test-slave-h6n0 krb5kdc[1102](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475000, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:03:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:03:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474615205622-30146-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474615205622-30146-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 19:03:20.500514 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474615205622-30146-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474615205622-30146-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474615205622-30146-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33413
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474615205622-30146-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33413
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:03:20.889107 1122 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:03:20.889395 1122 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:03:20.889557 1122 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:03:20.896910 1122 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:03:20.897037 1122 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:03:20.897133 1122 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:03:20.897241 1122 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:03:20.897310 1122 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:03:20.908582 1122 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474615205622-30146-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474615205622-30146-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33413
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474615205622-30146-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33413
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474615205622-30146-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:03:20.910408 1122 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:03:20.912338 1122 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:03:20.930075 1130 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:03:20.930277 1128 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:03:20.930236 1133 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:03:20.931109 1122 server_base.cc:1034] running on GCE node
I20260811 19:03:20.931999 1122 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:03:20.933535 1122 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:03:20.934818 1122 hybrid_clock.cc:648] HybridClock initialized: now 1786475000934656 us; error 119 us; skew 500 ppm
I20260811 19:03:20.935429 1122 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:03:20 dist-test-slave-h6n0 krb5kdc[1102](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475000, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:03:20.943972 1122 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 19:03:20.946276 1122 webserver.cc:460] Webserver started at http://127.0.0.1:33491/ using document root <none> and password file <none>
I20260811 19:03:20.947144 1122 fs_manager.cc:359] Metadata directory not provided
I20260811 19:03:20.947268 1122 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:03:20.947563 1122 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:03:20.950904 1122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474615205622-30146-0/minicluster-data/master-0/data/instance:
uuid: "9f44a657e88b4b4fb7ad198e8b4b23d6"
format_stamp: "Formatted at 2026-08-11 19:03:20 on dist-test-slave-h6n0"
server_key: "5c711fdbbceab2523b4f97d61777c25a"
server_key_iv: "74352c162ab34616df1741731a406082"
server_key_version: "encryptionkey@0"
I20260811 19:03:20.951673 1122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474615205622-30146-0/minicluster-data/master-0/wal/instance:
uuid: "9f44a657e88b4b4fb7ad198e8b4b23d6"
format_stamp: "Formatted at 2026-08-11 19:03:20 on dist-test-slave-h6n0"
server_key: "5c711fdbbceab2523b4f97d61777c25a"
server_key_iv: "74352c162ab34616df1741731a406082"
server_key_version: "encryptionkey@0"
I20260811 19:03:20.957898 1122 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260811 19:03:20.962865 1144 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:03:20.965077 1122 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260811 19:03:20.965318 1122 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474615205622-30146-0/minicluster-data/master-0/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474615205622-30146-0/minicluster-data/master-0/wal
uuid: "9f44a657e88b4b4fb7ad198e8b4b23d6"
format_stamp: "Formatted at 2026-08-11 19:03:20 on dist-test-slave-h6n0"
server_key: "5c711fdbbceab2523b4f97d61777c25a"
server_key_iv: "74352c162ab34616df1741731a406082"
server_key_version: "encryptionkey@0"
I20260811 19:03:20.965540 1122 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474615205622-30146-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474615205622-30146-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474615205622-30146-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:03:20.981410 1122 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:03:20.986150 1122 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:03:20.986474 1122 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:03:21.009789 1122 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33413
I20260811 19:03:21.009837 1236 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33413 every 8 connection(s)
I20260811 19:03:21.011369 1122 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
I20260811 19:03:21.015661 1237 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:03:21.018765 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 1122
I20260811 19:03:21.018990 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474615205622-30146-0/minicluster-data/master-0/wal/instance
I20260811 19:03:21.019518 30146 external_mini_cluster.cc:1428] Setting key 765b35f196c098781165bdfc3d5de870
I20260811 19:03:21.031020 1237 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f44a657e88b4b4fb7ad198e8b4b23d6: Bootstrap starting.
I20260811 19:03:21.035269 1237 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9f44a657e88b4b4fb7ad198e8b4b23d6: Neither blocks nor log segments found. Creating new log.
I20260811 19:03:21.037122 1237 log.cc:824] T 00000000000000000000000000000000 P 9f44a657e88b4b4fb7ad198e8b4b23d6: Log is configured to *not* fsync() on all Append() calls
I20260811 19:03:21.037932 1243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:21.022220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.52:36709 (local address 127.0.0.1:33413)
0811 19:03:21.023068 (+ 848us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:21.023080 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:21.023122 (+ 42us) server_negotiation.cc:407] Connection header received
0811 19:03:21.024780 (+ 1658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:21.024803 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:21.025223 (+ 420us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:21.025562 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:21.026690 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:21.028243 (+ 1553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:21.030955 (+ 2712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:21.031746 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:21.031895 (+ 149us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:21.033716 (+ 1821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:03:21.033729 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:03:21.033751 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:03:21.034793 (+ 1042us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:03:21.034930 (+ 137us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:03:21.034975 (+ 45us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:03:21.035211 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:21.035955 (+ 744us) server_negotiation.cc:299] Negotiation successful
0811 19:03:21.036285 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":624,"thread_start_us":403,"threads_started":1}
I20260811 19:03:21.041917 1237 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f44a657e88b4b4fb7ad198e8b4b23d6: No bootstrap required, opened a new log
I20260811 19:03:21.044644 30146 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 19:03:21.053174 1237 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9f44a657e88b4b4fb7ad198e8b4b23d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f44a657e88b4b4fb7ad198e8b4b23d6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33413 } }
I20260811 19:03:21.053681 1237 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9f44a657e88b4b4fb7ad198e8b4b23d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:03:21.053805 1237 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9f44a657e88b4b4fb7ad198e8b4b23d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f44a657e88b4b4fb7ad198e8b4b23d6, State: Initialized, Role: FOLLOWER
I20260811 19:03:21.054569 1237 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9f44a657e88b4b4fb7ad198e8b4b23d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f44a657e88b4b4fb7ad198e8b4b23d6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33413 } }
I20260811 19:03:21.054898 1237 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9f44a657e88b4b4fb7ad198e8b4b23d6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:03:21.055049 1237 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9f44a657e88b4b4fb7ad198e8b4b23d6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:03:21.055234 1237 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9f44a657e88b4b4fb7ad198e8b4b23d6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:03:21.059327 1243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:21.050656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.52:42805 (local address 127.0.0.1:33413)
0811 19:03:21.050928 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:21.050939 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:21.051480 (+ 541us) server_negotiation.cc:407] Connection header received
0811 19:03:21.051695 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:21.051700 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:21.051778 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:21.051891 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:21.052532 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:21.053407 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:21.055401 (+ 1994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:21.055947 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:21.056072 (+ 125us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:21.057753 (+ 1681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:03:21.057758 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:03:21.057761 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:03:21.057780 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:03:21.057928 (+ 148us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:03:21.057983 (+ 55us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:03:21.058112 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:21.058925 (+ 813us) server_negotiation.cc:299] Negotiation successful
0811 19:03:21.059002 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 19:03:21.059563 1237 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9f44a657e88b4b4fb7ad198e8b4b23d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f44a657e88b4b4fb7ad198e8b4b23d6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33413 } }
I20260811 19:03:21.060624 1237 leader_election.cc:302] T 00000000000000000000000000000000 P 9f44a657e88b4b4fb7ad198e8b4b23d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9f44a657e88b4b4fb7ad198e8b4b23d6; no voters:
I20260811 19:03:21.061559 1237 leader_election.cc:288] T 00000000000000000000000000000000 P 9f44a657e88b4b4fb7ad198e8b4b23d6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:03:21.061640 1259 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9f44a657e88b4b4fb7ad198e8b4b23d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:03:21.062417 1259 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9f44a657e88b4b4fb7ad198e8b4b23d6 [term 1 LEADER]: Becoming Leader. State: Replica: 9f44a657e88b4b4fb7ad198e8b4b23d6, State: Running, Role: LEADER
I20260811 19:03:21.063141 1259 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9f44a657e88b4b4fb7ad198e8b4b23d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f44a657e88b4b4fb7ad198e8b4b23d6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33413 } }
I20260811 19:03:21.064956 1237 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9f44a657e88b4b4fb7ad198e8b4b23d6 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:03:21.066334 1260 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f44a657e88b4b4fb7ad198e8b4b23d6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9f44a657e88b4b4fb7ad198e8b4b23d6. Latest consensus state: current_term: 1 leader_uuid: "9f44a657e88b4b4fb7ad198e8b4b23d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f44a657e88b4b4fb7ad198e8b4b23d6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33413 } } }
I20260811 19:03:21.066469 1262 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f44a657e88b4b4fb7ad198e8b4b23d6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9f44a657e88b4b4fb7ad198e8b4b23d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f44a657e88b4b4fb7ad198e8b4b23d6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33413 } } }
I20260811 19:03:21.066788 1260 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f44a657e88b4b4fb7ad198e8b4b23d6 [sys.catalog]: This master's current role is: LEADER
I20260811 19:03:21.066874 1262 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f44a657e88b4b4fb7ad198e8b4b23d6 [sys.catalog]: This master's current role is: LEADER
I20260811 19:03:21.067819 1268 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:03:21.074782 1268 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:03:21.088557 1268 catalog_manager.cc:1347] Generated new cluster ID: d5073af96c794d76b7e086e961746569
I20260811 19:03:21.088735 1268 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:03:21.121384 1268 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:03:21.123651 1268 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:03:21.142167 1268 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9f44a657e88b4b4fb7ad198e8b4b23d6: Generated new TSK 0
I20260811 19:03:21.143569 1268 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 19:03:21.167569 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 1122
W20260811 19:03:21.169376 1227 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.52:36709
W20260811 19:03:21.170423 30146 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 1122: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 19:03:21.170642 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 1122
W20260811 19:03:21.201215 1100 connection.cc:535] client connection to 127.0.0.1:33413 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33413): Connection reset by peer (error 104)
2026-08-11T19:03:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (2542 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474615205622-30146-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:03:21 dist-test-slave-h6n0 krb5kdc[1300](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:03:21 dist-test-slave-h6n0 krb5kdc[1300](info): set up 2 sockets
Aug 11 19:03:21 dist-test-slave-h6n0 krb5kdc[1300](info): commencing operation
krb5kdc: starting...
W20260811 19:03:22.670449 30146 mini_kdc.cc:121] Time spent starting KDC: real 1.449s user 0.000s sys 0.053s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474615205622-30146-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474615205622-30146-0/krb5kdc/test-user.keytab.
Aug 11 19:03:22 dist-test-slave-h6n0 krb5kdc[1300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475002, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:03:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:03:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474615205622-30146-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474615205622-30146-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 19:03:23.025768 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474615205622-30146-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474615205622-30146-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474615205622-30146-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39247
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474615205622-30146-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39247
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:03:23.408028 1323 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:03:23.408384 1323 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:03:23.408486 1323 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:03:23.418406 1323 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:03:23.418682 1323 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:03:23.418761 1323 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:03:23.418845 1323 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:03:23.418900 1323 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:03:23.432039 1323 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474615205622-30146-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474615205622-30146-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39247
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474615205622-30146-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39247
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474615205622-30146-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:03:23.433887 1323 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:03:23.435312 1323 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:03:23.451097 1335 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:03:23.451053 1333 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:03:23.451159 1332 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:03:23.454140 1323 server_base.cc:1034] running on GCE node
I20260811 19:03:23.455291 1323 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:03:23.456985 1323 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:03:23.458443 1323 hybrid_clock.cc:648] HybridClock initialized: now 1786475003458366 us; error 51 us; skew 500 ppm
I20260811 19:03:23.459115 1323 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:03:23 dist-test-slave-h6n0 krb5kdc[1300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475003, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:03:23.465183 1323 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 19:03:23.469393 1323 webserver.cc:460] Webserver started at http://127.0.0.1:38291/ using document root <none> and password file <none>
I20260811 19:03:23.471465 1323 fs_manager.cc:359] Metadata directory not provided
I20260811 19:03:23.472103 1323 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:03:23.472827 1323 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:03:23.479194 1323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474615205622-30146-0/minicluster-data/master-0/data/instance:
uuid: "f4a8bc4c072043488e7954806973a5d0"
format_stamp: "Formatted at 2026-08-11 19:03:23 on dist-test-slave-h6n0"
server_key: "56cec64789f9a33fe384168a33efd5c4"
server_key_iv: "4c013fb3d615dd948456a93144dd15d4"
server_key_version: "encryptionkey@0"
I20260811 19:03:23.480824 1323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474615205622-30146-0/minicluster-data/master-0/wal/instance:
uuid: "f4a8bc4c072043488e7954806973a5d0"
format_stamp: "Formatted at 2026-08-11 19:03:23 on dist-test-slave-h6n0"
server_key: "56cec64789f9a33fe384168a33efd5c4"
server_key_iv: "4c013fb3d615dd948456a93144dd15d4"
server_key_version: "encryptionkey@0"
I20260811 19:03:23.489185 1323 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.003s sys 0.003s
I20260811 19:03:23.495757 1348 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:03:23.497864 1323 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.005s sys 0.000s
I20260811 19:03:23.498153 1323 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474615205622-30146-0/minicluster-data/master-0/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474615205622-30146-0/minicluster-data/master-0/wal
uuid: "f4a8bc4c072043488e7954806973a5d0"
format_stamp: "Formatted at 2026-08-11 19:03:23 on dist-test-slave-h6n0"
server_key: "56cec64789f9a33fe384168a33efd5c4"
server_key_iv: "4c013fb3d615dd948456a93144dd15d4"
server_key_version: "encryptionkey@0"
I20260811 19:03:23.498492 1323 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474615205622-30146-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474615205622-30146-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474615205622-30146-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:03:23.510326 1323 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:03:23.516723 1323 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:03:23.517524 1323 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:03:23.539399 1323 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39247
I20260811 19:03:23.539451 1436 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39247 every 8 connection(s)
I20260811 19:03:23.540839 1323 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
I20260811 19:03:23.544442 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 1323
I20260811 19:03:23.544641 1437 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:03:23.544660 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474615205622-30146-0/minicluster-data/master-0/wal/instance
I20260811 19:03:23.545150 30146 external_mini_cluster.cc:1428] Setting key 7ce4ec6da3d38915c9ae3ca019c5ffee
I20260811 19:03:23.555397 1442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:23.547164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.52:33719 (local address 127.0.0.1:39247)
0811 19:03:23.548114 (+ 950us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:23.548130 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:23.548186 (+ 56us) server_negotiation.cc:407] Connection header received
0811 19:03:23.550059 (+ 1873us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:23.550093 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:23.550567 (+ 474us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:23.551028 (+ 461us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:23.552281 (+ 1253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:03:23.552294 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:03:23.552313 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:03:23.552349 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:03:23.552433 (+ 84us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:03:23.552476 (+ 43us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:03:23.552719 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:23.553738 (+ 1019us) server_negotiation.cc:299] Negotiation successful
0811 19:03:23.553971 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":657,"thread_start_us":411,"threads_started":1}
I20260811 19:03:23.563447 1437 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4a8bc4c072043488e7954806973a5d0: Bootstrap starting.
I20260811 19:03:23.563874 30146 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 19:03:23.567442 1437 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f4a8bc4c072043488e7954806973a5d0: Neither blocks nor log segments found. Creating new log.
I20260811 19:03:23.568751 1437 log.cc:824] T 00000000000000000000000000000000 P f4a8bc4c072043488e7954806973a5d0: Log is configured to *not* fsync() on all Append() calls
I20260811 19:03:23.572569 1437 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4a8bc4c072043488e7954806973a5d0: No bootstrap required, opened a new log
I20260811 19:03:23.572818 1442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:23.570008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.52:54257 (local address 127.0.0.1:39247)
0811 19:03:23.570176 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:23.570181 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:23.570763 (+ 582us) server_negotiation.cc:407] Connection header received
0811 19:03:23.570910 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:23.570915 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:23.570973 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:23.571106 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:23.572106 (+ 1000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:03:23.572112 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:03:23.572116 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:03:23.572134 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:03:23.572181 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:03:23.572204 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:03:23.572327 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:23.572415 (+ 88us) server_negotiation.cc:299] Negotiation successful
0811 19:03:23.572473 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260811 19:03:23.580289 1437 raft_consensus.cc:348] T 00000000000000000000000000000000 P f4a8bc4c072043488e7954806973a5d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4a8bc4c072043488e7954806973a5d0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39247 } }
I20260811 19:03:23.580729 1437 raft_consensus.cc:374] T 00000000000000000000000000000000 P f4a8bc4c072043488e7954806973a5d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:03:23.580802 1437 raft_consensus.cc:729] T 00000000000000000000000000000000 P f4a8bc4c072043488e7954806973a5d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4a8bc4c072043488e7954806973a5d0, State: Initialized, Role: FOLLOWER
I20260811 19:03:23.581364 1437 consensus_queue.cc:260] T 00000000000000000000000000000000 P f4a8bc4c072043488e7954806973a5d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4a8bc4c072043488e7954806973a5d0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39247 } }
I20260811 19:03:23.581579 1437 raft_consensus.cc:388] T 00000000000000000000000000000000 P f4a8bc4c072043488e7954806973a5d0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:03:23.581701 1437 raft_consensus.cc:482] T 00000000000000000000000000000000 P f4a8bc4c072043488e7954806973a5d0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:03:23.581831 1437 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f4a8bc4c072043488e7954806973a5d0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:03:23.584489 1437 raft_consensus.cc:504] T 00000000000000000000000000000000 P f4a8bc4c072043488e7954806973a5d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4a8bc4c072043488e7954806973a5d0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39247 } }
I20260811 19:03:23.584967 1437 leader_election.cc:302] T 00000000000000000000000000000000 P f4a8bc4c072043488e7954806973a5d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f4a8bc4c072043488e7954806973a5d0; no voters:
I20260811 19:03:23.585597 1437 leader_election.cc:288] T 00000000000000000000000000000000 P f4a8bc4c072043488e7954806973a5d0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:03:23.586043 1455 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f4a8bc4c072043488e7954806973a5d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:03:23.587004 1455 raft_consensus.cc:686] T 00000000000000000000000000000000 P f4a8bc4c072043488e7954806973a5d0 [term 1 LEADER]: Becoming Leader. State: Replica: f4a8bc4c072043488e7954806973a5d0, State: Running, Role: LEADER
I20260811 19:03:23.587577 1455 consensus_queue.cc:237] T 00000000000000000000000000000000 P f4a8bc4c072043488e7954806973a5d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4a8bc4c072043488e7954806973a5d0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39247 } }
I20260811 19:03:23.588286 1437 sys_catalog.cc:564] T 00000000000000000000000000000000 P f4a8bc4c072043488e7954806973a5d0 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:03:23.590273 1458 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4a8bc4c072043488e7954806973a5d0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f4a8bc4c072043488e7954806973a5d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4a8bc4c072043488e7954806973a5d0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39247 } } }
I20260811 19:03:23.590674 1458 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4a8bc4c072043488e7954806973a5d0 [sys.catalog]: This master's current role is: LEADER
I20260811 19:03:23.590745 1463 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4a8bc4c072043488e7954806973a5d0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f4a8bc4c072043488e7954806973a5d0. Latest consensus state: current_term: 1 leader_uuid: "f4a8bc4c072043488e7954806973a5d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4a8bc4c072043488e7954806973a5d0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39247 } } }
I20260811 19:03:23.591069 1463 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4a8bc4c072043488e7954806973a5d0 [sys.catalog]: This master's current role is: LEADER
I20260811 19:03:23.599295 1466 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:03:23.605141 1466 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:03:23.615394 1466 catalog_manager.cc:1347] Generated new cluster ID: 5ca25d7ee03d48aeb377e03fecfdf4c5
I20260811 19:03:23.615576 1466 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:03:23.625613 1466 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:03:23.626029 1466 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:03:23.636358 1466 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f4a8bc4c072043488e7954806973a5d0: Generated new TSK 0
I20260811 19:03:23.637260 1466 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 19:03:23.652091 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 1323
W20260811 19:03:23.653324 1431 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.52:33719
W20260811 19:03:23.654203 30146 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 1323: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 19:03:23.654412 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 1323
2026-08-11T19:03:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (2480 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474615205622-30146-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:03:23 dist-test-slave-h6n0 krb5kdc[1497](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:03:23 dist-test-slave-h6n0 krb5kdc[1497](info): set up 2 sockets
Aug 11 19:03:23 dist-test-slave-h6n0 krb5kdc[1497](info): commencing operation
krb5kdc: starting...
W20260811 19:03:25.108426 30146 mini_kdc.cc:121] Time spent starting KDC: real 1.407s user 0.004s sys 0.061s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474615205622-30146-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474615205622-30146-0/krb5kdc/test-user.keytab.
Aug 11 19:03:25 dist-test-slave-h6n0 krb5kdc[1497](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475005, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:03:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:03:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474615205622-30146-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474615205622-30146-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 19:03:25.470753 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474615205622-30146-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474615205622-30146-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474615205622-30146-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46087
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474615205622-30146-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:32853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46087
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:03:25.836118 1520 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:03:25.836441 1520 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:03:25.836629 1520 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:03:25.844946 1520 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:03:25.845079 1520 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:03:25.845130 1520 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:03:25.845279 1520 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:03:25.845582 1520 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:03:25.856925 1520 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:32853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474615205622-30146-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474615205622-30146-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46087
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474615205622-30146-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46087
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474615205622-30146-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:03:25.858340 1520 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:03:25.859740 1520 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:03:25.872745 1527 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:03:25.872709 1526 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:03:25.872797 1530 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:03:25.873539 1520 server_base.cc:1034] running on GCE node
I20260811 19:03:25.874491 1520 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:03:25.875991 1520 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:03:25.877229 1520 hybrid_clock.cc:648] HybridClock initialized: now 1786475005877175 us; error 36 us; skew 500 ppm
I20260811 19:03:25.877732 1520 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:03:25 dist-test-slave-h6n0 krb5kdc[1497](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475005, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:03:25.881942 1520 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 19:03:25.884552 1520 webserver.cc:460] Webserver started at http://127.0.0.1:38881/ using document root <none> and password file <none>
I20260811 19:03:25.885324 1520 fs_manager.cc:359] Metadata directory not provided
I20260811 19:03:25.885509 1520 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:03:25.885852 1520 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:03:25.889225 1520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474615205622-30146-0/minicluster-data/master-0/data/instance:
uuid: "d0fe8fa676da40b2bcbe4371961549cd"
format_stamp: "Formatted at 2026-08-11 19:03:25 on dist-test-slave-h6n0"
server_key: "4a6d6b661cb5244e5226b68d131b86d8"
server_key_iv: "cf44469e7c1b6df34ce5340d272b2bd8"
server_key_version: "encryptionkey@0"
I20260811 19:03:25.890199 1520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474615205622-30146-0/minicluster-data/master-0/wal/instance:
uuid: "d0fe8fa676da40b2bcbe4371961549cd"
format_stamp: "Formatted at 2026-08-11 19:03:25 on dist-test-slave-h6n0"
server_key: "4a6d6b661cb5244e5226b68d131b86d8"
server_key_iv: "cf44469e7c1b6df34ce5340d272b2bd8"
server_key_version: "encryptionkey@0"
I20260811 19:03:25.896107 1520 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 19:03:25.899926 1538 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:03:25.901417 1520 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 19:03:25.901731 1520 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474615205622-30146-0/minicluster-data/master-0/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474615205622-30146-0/minicluster-data/master-0/wal
uuid: "d0fe8fa676da40b2bcbe4371961549cd"
format_stamp: "Formatted at 2026-08-11 19:03:25 on dist-test-slave-h6n0"
server_key: "4a6d6b661cb5244e5226b68d131b86d8"
server_key_iv: "cf44469e7c1b6df34ce5340d272b2bd8"
server_key_version: "encryptionkey@0"
I20260811 19:03:25.901998 1520 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474615205622-30146-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474615205622-30146-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474615205622-30146-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:03:25.930513 1520 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:03:25.936483 1520 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:03:25.936756 1520 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:03:25.957877 1520 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46087
I20260811 19:03:25.958011 1636 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46087 every 8 connection(s)
I20260811 19:03:25.959609 1520 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
I20260811 19:03:25.961361 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 1520
I20260811 19:03:25.961565 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474615205622-30146-0/minicluster-data/master-0/wal/instance
I20260811 19:03:25.962038 30146 external_mini_cluster.cc:1428] Setting key 6047414c369f0e64780c9ca73931acf2
I20260811 19:03:25.964576 1639 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:03:25.977334 1639 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0fe8fa676da40b2bcbe4371961549cd: Bootstrap starting.
I20260811 19:03:25.981386 1639 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d0fe8fa676da40b2bcbe4371961549cd: Neither blocks nor log segments found. Creating new log.
Aug 11 19:03:25 dist-test-slave-h6n0 krb5kdc[1497](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475005, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 19:03:25.984078 1639 log.cc:824] T 00000000000000000000000000000000 P d0fe8fa676da40b2bcbe4371961549cd: Log is configured to *not* fsync() on all Append() calls
I20260811 19:03:25.988673 1639 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0fe8fa676da40b2bcbe4371961549cd: No bootstrap required, opened a new log
I20260811 19:03:25.995358 1647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:25.963893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.52:60455 (local address 127.0.0.1:46087)
0811 19:03:25.964718 (+ 825us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:25.964736 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:25.964792 (+ 56us) server_negotiation.cc:407] Connection header received
0811 19:03:25.966583 (+ 1791us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:25.966618 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:25.967025 (+ 407us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:25.967355 (+ 330us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:25.969255 (+ 1900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:25.970714 (+ 1459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:25.972566 (+ 1852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:25.973352 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:25.973524 (+ 172us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:25.984247 (+ 10723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:03:25.984266 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:03:25.984296 (+ 30us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:03:25.984378 (+ 82us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:03:25.989315 (+ 4937us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:25.990284 (+ 969us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:25.990293 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:25.990301 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:25.990557 (+ 256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:25.991178 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:25.991184 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:25.991191 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:25.991935 (+ 744us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:03:25.992305 (+ 370us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:25.993512 (+ 1207us) server_negotiation.cc:299] Negotiation successful
0811 19:03:25.993882 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":583,"thread_start_us":373,"threads_started":1}
I20260811 19:03:25.998993 1639 raft_consensus.cc:348] T 00000000000000000000000000000000 P d0fe8fa676da40b2bcbe4371961549cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0fe8fa676da40b2bcbe4371961549cd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46087 } }
I20260811 19:03:25.999408 1639 raft_consensus.cc:374] T 00000000000000000000000000000000 P d0fe8fa676da40b2bcbe4371961549cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:03:25.999553 1639 raft_consensus.cc:729] T 00000000000000000000000000000000 P d0fe8fa676da40b2bcbe4371961549cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0fe8fa676da40b2bcbe4371961549cd, State: Initialized, Role: FOLLOWER
I20260811 19:03:26.000741 30146 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 19:03:26.000905 1639 consensus_queue.cc:260] T 00000000000000000000000000000000 P d0fe8fa676da40b2bcbe4371961549cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0fe8fa676da40b2bcbe4371961549cd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46087 } }
I20260811 19:03:26.001662 1639 raft_consensus.cc:388] T 00000000000000000000000000000000 P d0fe8fa676da40b2bcbe4371961549cd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:03:26.001933 1639 raft_consensus.cc:482] T 00000000000000000000000000000000 P d0fe8fa676da40b2bcbe4371961549cd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:03:26.002216 1639 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d0fe8fa676da40b2bcbe4371961549cd [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:03:26.005293 1639 raft_consensus.cc:504] T 00000000000000000000000000000000 P d0fe8fa676da40b2bcbe4371961549cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0fe8fa676da40b2bcbe4371961549cd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46087 } }
I20260811 19:03:26.006069 1639 leader_election.cc:302] T 00000000000000000000000000000000 P d0fe8fa676da40b2bcbe4371961549cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d0fe8fa676da40b2bcbe4371961549cd; no voters:
I20260811 19:03:26.006894 1639 leader_election.cc:288] T 00000000000000000000000000000000 P d0fe8fa676da40b2bcbe4371961549cd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:03:26.007004 1656 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d0fe8fa676da40b2bcbe4371961549cd [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:03:26.009565 1639 sys_catalog.cc:564] T 00000000000000000000000000000000 P d0fe8fa676da40b2bcbe4371961549cd [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:03:26.010828 1656 raft_consensus.cc:686] T 00000000000000000000000000000000 P d0fe8fa676da40b2bcbe4371961549cd [term 1 LEADER]: Becoming Leader. State: Replica: d0fe8fa676da40b2bcbe4371961549cd, State: Running, Role: LEADER
I20260811 19:03:26.011636 1656 consensus_queue.cc:237] T 00000000000000000000000000000000 P d0fe8fa676da40b2bcbe4371961549cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0fe8fa676da40b2bcbe4371961549cd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46087 } }
I20260811 19:03:26.023864 1659 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0fe8fa676da40b2bcbe4371961549cd [sys.catalog]: SysCatalogTable state changed. Reason: New leader d0fe8fa676da40b2bcbe4371961549cd. Latest consensus state: current_term: 1 leader_uuid: "d0fe8fa676da40b2bcbe4371961549cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0fe8fa676da40b2bcbe4371961549cd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46087 } } }
I20260811 19:03:26.024379 1659 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0fe8fa676da40b2bcbe4371961549cd [sys.catalog]: This master's current role is: LEADER
I20260811 19:03:26.025386 1657 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0fe8fa676da40b2bcbe4371961549cd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d0fe8fa676da40b2bcbe4371961549cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0fe8fa676da40b2bcbe4371961549cd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46087 } } }
I20260811 19:03:26.026032 1657 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0fe8fa676da40b2bcbe4371961549cd [sys.catalog]: This master's current role is: LEADER
I20260811 19:03:26.030750 1647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:26.005928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.52:48793 (local address 127.0.0.1:46087)
0811 19:03:26.006187 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:26.006196 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:26.006744 (+ 548us) server_negotiation.cc:407] Connection header received
0811 19:03:26.006966 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:26.006974 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:26.007121 (+ 147us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:26.007231 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:26.009308 (+ 2077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:26.010376 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:26.012700 (+ 2324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:26.013160 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:26.013302 (+ 142us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:26.016899 (+ 3597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:03:26.016905 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:03:26.016909 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:03:26.016943 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:03:26.021516 (+ 4573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:26.022513 (+ 997us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:26.022518 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:26.022522 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:26.022769 (+ 247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:26.027491 (+ 4722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:26.027502 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:26.027509 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:26.028192 (+ 683us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:03:26.028656 (+ 464us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:26.029843 (+ 1187us) server_negotiation.cc:299] Negotiation successful
0811 19:03:26.030194 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
I20260811 19:03:26.031471 1679 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:03:26.038796 1679 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:03:26.048256 1679 catalog_manager.cc:1347] Generated new cluster ID: 6f95213d292a4c768c303a8ade19f534
I20260811 19:03:26.048395 1679 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:03:26.070708 1679 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:03:26.073285 1679 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:03:26.087328 1679 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d0fe8fa676da40b2bcbe4371961549cd: Generated new TSK 0
I20260811 19:03:26.088255 1679 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 19:03:26.100827 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 1520
I20260811 19:03:26.102159 1631 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:03:26.203701 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 1520
W20260811 19:03:26.231024 1495 connection.cc:535] client connection to 127.0.0.1:46087 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46087): Connection reset by peer (error 104)
2026-08-11T19:03:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2551 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474615205622-30146-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:03:26 dist-test-slave-h6n0 krb5kdc[1695](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:03:26 dist-test-slave-h6n0 krb5kdc[1695](info): set up 2 sockets
Aug 11 19:03:26 dist-test-slave-h6n0 krb5kdc[1695](info): commencing operation
krb5kdc: starting...
W20260811 19:03:27.638374 30146 mini_kdc.cc:121] Time spent starting KDC: real 1.385s user 0.000s sys 0.055s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474615205622-30146-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474615205622-30146-0/krb5kdc/test-user.keytab.
Aug 11 19:03:27 dist-test-slave-h6n0 krb5kdc[1695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475007, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:03:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:03:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474615205622-30146-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474615205622-30146-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 19:03:27.999029 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474615205622-30146-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474615205622-30146-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474615205622-30146-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39251
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474615205622-30146-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39251
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:03:28.421559 1719 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:03:28.421885 1719 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:03:28.422029 1719 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:03:28.429757 1719 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:03:28.429900 1719 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:03:28.429986 1719 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:03:28.430119 1719 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:03:28.430217 1719 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:03:28.441596 1719 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474615205622-30146-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474615205622-30146-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39251
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474615205622-30146-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39251
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474615205622-30146-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:03:28.443162 1719 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:03:28.444618 1719 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:03:28.460038 1730 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:03:28.460067 1732 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:03:28.460117 1735 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:03:28.461733 1719 server_base.cc:1034] running on GCE node
I20260811 19:03:28.462812 1719 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:03:28.465013 1719 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:03:28.466321 1719 hybrid_clock.cc:648] HybridClock initialized: now 1786475008466277 us; error 32 us; skew 500 ppm
I20260811 19:03:28.466892 1719 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:03:28 dist-test-slave-h6n0 krb5kdc[1695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475008, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:03:28.472662 1719 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 19:03:28.474838 1719 webserver.cc:460] Webserver started at http://127.0.0.1:43775/ using document root <none> and password file <none>
I20260811 19:03:28.475726 1719 fs_manager.cc:359] Metadata directory not provided
I20260811 19:03:28.475912 1719 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:03:28.476876 1719 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:03:28.481745 1719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474615205622-30146-0/minicluster-data/master-0/data/instance:
uuid: "485f6d1679fc4552a186a3a6d8f2d80e"
format_stamp: "Formatted at 2026-08-11 19:03:28 on dist-test-slave-h6n0"
server_key: "03c9a24c5c1b3b0d29e90819e18fdf13"
server_key_iv: "7740c6746a764113f0202198efee5b14"
server_key_version: "encryptionkey@0"
I20260811 19:03:28.482785 1719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474615205622-30146-0/minicluster-data/master-0/wal/instance:
uuid: "485f6d1679fc4552a186a3a6d8f2d80e"
format_stamp: "Formatted at 2026-08-11 19:03:28 on dist-test-slave-h6n0"
server_key: "03c9a24c5c1b3b0d29e90819e18fdf13"
server_key_iv: "7740c6746a764113f0202198efee5b14"
server_key_version: "encryptionkey@0"
I20260811 19:03:28.488883 1719 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 19:03:28.493564 1742 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:03:28.495488 1719 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260811 19:03:28.495729 1719 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474615205622-30146-0/minicluster-data/master-0/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474615205622-30146-0/minicluster-data/master-0/wal
uuid: "485f6d1679fc4552a186a3a6d8f2d80e"
format_stamp: "Formatted at 2026-08-11 19:03:28 on dist-test-slave-h6n0"
server_key: "03c9a24c5c1b3b0d29e90819e18fdf13"
server_key_iv: "7740c6746a764113f0202198efee5b14"
server_key_version: "encryptionkey@0"
I20260811 19:03:28.495975 1719 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474615205622-30146-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474615205622-30146-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474615205622-30146-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:03:28.520663 1719 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:03:28.527081 1719 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:03:28.527490 1719 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:03:28.556691 1719 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39251
I20260811 19:03:28.556808 1841 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39251 every 8 connection(s)
I20260811 19:03:28.558375 1719 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
I20260811 19:03:28.559144 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 1719
I20260811 19:03:28.559374 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474615205622-30146-0/minicluster-data/master-0/wal/instance
I20260811 19:03:28.559962 30146 external_mini_cluster.cc:1428] Setting key 29e388667631112703c32233cba5f539
I20260811 19:03:28.564368 1842 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:03:28.576678 1845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:28.562126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.52:45019 (local address 127.0.0.1:39251)
0811 19:03:28.563103 (+ 977us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:28.563117 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:28.563163 (+ 46us) server_negotiation.cc:407] Connection header received
0811 19:03:28.565300 (+ 2137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:28.565325 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:28.565787 (+ 462us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:28.566217 (+ 430us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:28.566851 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:28.568633 (+ 1782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:28.570453 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:28.571430 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:28.571674 (+ 244us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:28.573529 (+ 1855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:03:28.573539 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:03:28.573559 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:03:28.574337 (+ 778us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:03:28.574447 (+ 110us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:03:28.574481 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:03:28.574847 (+ 366us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:28.575143 (+ 296us) server_negotiation.cc:299] Negotiation successful
0811 19:03:28.575381 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":738,"thread_start_us":488,"threads_started":1}
I20260811 19:03:28.577772 1842 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 485f6d1679fc4552a186a3a6d8f2d80e: Bootstrap starting.
I20260811 19:03:28.582314 30146 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 19:03:28.583362 1842 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 485f6d1679fc4552a186a3a6d8f2d80e: Neither blocks nor log segments found. Creating new log.
I20260811 19:03:28.585428 1842 log.cc:824] T 00000000000000000000000000000000 P 485f6d1679fc4552a186a3a6d8f2d80e: Log is configured to *not* fsync() on all Append() calls
I20260811 19:03:28.590459 1842 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 485f6d1679fc4552a186a3a6d8f2d80e: No bootstrap required, opened a new log
I20260811 19:03:28.598394 1845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:28.590036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.52:48753 (local address 127.0.0.1:39251)
0811 19:03:28.590322 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:28.590333 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:28.591078 (+ 745us) server_negotiation.cc:407] Connection header received
0811 19:03:28.591349 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:28.591353 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:28.591433 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:28.591548 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:28.592254 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:28.593326 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:28.594830 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:28.595337 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:28.595522 (+ 185us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:28.597435 (+ 1913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:03:28.597441 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:03:28.597444 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:03:28.597468 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:03:28.597540 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:03:28.597559 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:03:28.597731 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:28.597946 (+ 215us) server_negotiation.cc:299] Negotiation successful
0811 19:03:28.598010 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 19:03:28.601617 1842 raft_consensus.cc:348] T 00000000000000000000000000000000 P 485f6d1679fc4552a186a3a6d8f2d80e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "485f6d1679fc4552a186a3a6d8f2d80e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39251 } }
I20260811 19:03:28.602242 1842 raft_consensus.cc:374] T 00000000000000000000000000000000 P 485f6d1679fc4552a186a3a6d8f2d80e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:03:28.602418 1842 raft_consensus.cc:729] T 00000000000000000000000000000000 P 485f6d1679fc4552a186a3a6d8f2d80e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 485f6d1679fc4552a186a3a6d8f2d80e, State: Initialized, Role: FOLLOWER
I20260811 19:03:28.603240 1842 consensus_queue.cc:260] T 00000000000000000000000000000000 P 485f6d1679fc4552a186a3a6d8f2d80e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "485f6d1679fc4552a186a3a6d8f2d80e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39251 } }
I20260811 19:03:28.603550 1842 raft_consensus.cc:388] T 00000000000000000000000000000000 P 485f6d1679fc4552a186a3a6d8f2d80e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:03:28.603720 1842 raft_consensus.cc:482] T 00000000000000000000000000000000 P 485f6d1679fc4552a186a3a6d8f2d80e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:03:28.603986 1842 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 485f6d1679fc4552a186a3a6d8f2d80e [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:03:28.606808 1842 raft_consensus.cc:504] T 00000000000000000000000000000000 P 485f6d1679fc4552a186a3a6d8f2d80e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "485f6d1679fc4552a186a3a6d8f2d80e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39251 } }
I20260811 19:03:28.607434 1842 leader_election.cc:302] T 00000000000000000000000000000000 P 485f6d1679fc4552a186a3a6d8f2d80e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 485f6d1679fc4552a186a3a6d8f2d80e; no voters:
I20260811 19:03:28.608201 1842 leader_election.cc:288] T 00000000000000000000000000000000 P 485f6d1679fc4552a186a3a6d8f2d80e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:03:28.608319 1856 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 485f6d1679fc4552a186a3a6d8f2d80e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:03:28.609272 1856 raft_consensus.cc:686] T 00000000000000000000000000000000 P 485f6d1679fc4552a186a3a6d8f2d80e [term 1 LEADER]: Becoming Leader. State: Replica: 485f6d1679fc4552a186a3a6d8f2d80e, State: Running, Role: LEADER
I20260811 19:03:28.609910 1856 consensus_queue.cc:237] T 00000000000000000000000000000000 P 485f6d1679fc4552a186a3a6d8f2d80e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "485f6d1679fc4552a186a3a6d8f2d80e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39251 } }
I20260811 19:03:28.610989 1842 sys_catalog.cc:564] T 00000000000000000000000000000000 P 485f6d1679fc4552a186a3a6d8f2d80e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:03:28.614415 1857 sys_catalog.cc:455] T 00000000000000000000000000000000 P 485f6d1679fc4552a186a3a6d8f2d80e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "485f6d1679fc4552a186a3a6d8f2d80e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "485f6d1679fc4552a186a3a6d8f2d80e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39251 } } }
I20260811 19:03:28.614246 1858 sys_catalog.cc:455] T 00000000000000000000000000000000 P 485f6d1679fc4552a186a3a6d8f2d80e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 485f6d1679fc4552a186a3a6d8f2d80e. Latest consensus state: current_term: 1 leader_uuid: "485f6d1679fc4552a186a3a6d8f2d80e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "485f6d1679fc4552a186a3a6d8f2d80e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39251 } } }
I20260811 19:03:28.614871 1857 sys_catalog.cc:458] T 00000000000000000000000000000000 P 485f6d1679fc4552a186a3a6d8f2d80e [sys.catalog]: This master's current role is: LEADER
I20260811 19:03:28.615000 1858 sys_catalog.cc:458] T 00000000000000000000000000000000 P 485f6d1679fc4552a186a3a6d8f2d80e [sys.catalog]: This master's current role is: LEADER
I20260811 19:03:28.618757 1867 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:03:28.624723 1867 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:03:28.638870 1867 catalog_manager.cc:1347] Generated new cluster ID: bc8842fb97aa4eb4a9e00d2b73bf36a0
I20260811 19:03:28.639149 1867 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:03:28.648336 1867 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:03:28.650252 1867 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:03:28.662009 1867 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 485f6d1679fc4552a186a3a6d8f2d80e: Generated new TSK 0
I20260811 19:03:28.663199 1867 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 19:03:28.683672 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 1719
W20260811 19:03:28.685457 1830 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.52:45019
W20260811 19:03:28.686348 30146 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 1719: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 19:03:28.686491 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 1719
W20260811 19:03:28.712647 1693 connection.cc:535] client connection to 127.0.0.1:39251 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39251): Connection reset by peer (error 104)
2026-08-11T19:03:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (2480 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474615205622-30146-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:03:28 dist-test-slave-h6n0 krb5kdc[1894](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:03:28 dist-test-slave-h6n0 krb5kdc[1894](info): set up 2 sockets
Aug 11 19:03:28 dist-test-slave-h6n0 krb5kdc[1894](info): commencing operation
krb5kdc: starting...
W20260811 19:03:30.125635 30146 mini_kdc.cc:121] Time spent starting KDC: real 1.392s user 0.003s sys 0.050s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474615205622-30146-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474615205622-30146-0/krb5kdc/test-user.keytab.
Aug 11 19:03:30 dist-test-slave-h6n0 krb5kdc[1894](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475010, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:03:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:03:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474615205622-30146-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474615205622-30146-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 19:03:30.496104 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474615205622-30146-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474615205622-30146-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474615205622-30146-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33995
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474615205622-30146-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33995
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:03:30.881910 1920 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:03:30.882339 1920 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:03:30.882580 1920 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:03:30.892755 1920 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:03:30.892925 1920 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:03:30.892984 1920 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:03:30.893165 1920 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:03:30.893260 1920 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:03:30.904026 1920 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474615205622-30146-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474615205622-30146-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33995
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474615205622-30146-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33995
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474615205622-30146-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:03:30.905462 1920 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:03:30.906952 1920 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:03:30.921166 1938 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:03:30.921221 1930 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:03:30.921180 1934 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:03:30.922791 1920 server_base.cc:1034] running on GCE node
I20260811 19:03:30.923635 1920 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:03:30.925124 1920 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:03:30.926414 1920 hybrid_clock.cc:648] HybridClock initialized: now 1786475010926352 us; error 53 us; skew 500 ppm
I20260811 19:03:30.926859 1920 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:03:30 dist-test-slave-h6n0 krb5kdc[1894](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475010, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:03:30.931052 1920 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 19:03:30.933105 1920 webserver.cc:460] Webserver started at http://127.0.0.1:42177/ using document root <none> and password file <none>
I20260811 19:03:30.933858 1920 fs_manager.cc:359] Metadata directory not provided
I20260811 19:03:30.934022 1920 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:03:30.934373 1920 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:03:30.939042 1920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474615205622-30146-0/minicluster-data/master-0/data/instance:
uuid: "afab13f2822b4a56bdfb4093c6dad43b"
format_stamp: "Formatted at 2026-08-11 19:03:30 on dist-test-slave-h6n0"
server_key: "d309a3d97551c5f81342dad19193d307"
server_key_iv: "6f303d6023e4ff0f4f13ecb07fa32782"
server_key_version: "encryptionkey@0"
I20260811 19:03:30.940199 1920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474615205622-30146-0/minicluster-data/master-0/wal/instance:
uuid: "afab13f2822b4a56bdfb4093c6dad43b"
format_stamp: "Formatted at 2026-08-11 19:03:30 on dist-test-slave-h6n0"
server_key: "d309a3d97551c5f81342dad19193d307"
server_key_iv: "6f303d6023e4ff0f4f13ecb07fa32782"
server_key_version: "encryptionkey@0"
I20260811 19:03:30.951251 1920 fs_manager.cc:691] Time spent creating directory manager: real 0.011s user 0.004s sys 0.007s
I20260811 19:03:30.955705 1948 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:03:30.957489 1920 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 19:03:30.957705 1920 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474615205622-30146-0/minicluster-data/master-0/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474615205622-30146-0/minicluster-data/master-0/wal
uuid: "afab13f2822b4a56bdfb4093c6dad43b"
format_stamp: "Formatted at 2026-08-11 19:03:30 on dist-test-slave-h6n0"
server_key: "d309a3d97551c5f81342dad19193d307"
server_key_iv: "6f303d6023e4ff0f4f13ecb07fa32782"
server_key_version: "encryptionkey@0"
I20260811 19:03:30.957947 1920 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474615205622-30146-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474615205622-30146-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474615205622-30146-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:03:30.964536 1920 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:03:30.969183 1920 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:03:30.969521 1920 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:03:30.991537 1920 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33995
I20260811 19:03:30.991588 2035 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33995 every 8 connection(s)
I20260811 19:03:30.993052 1920 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
I20260811 19:03:30.996927 2036 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:03:30.999539 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 1920
I20260811 19:03:30.999763 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474615205622-30146-0/minicluster-data/master-0/wal/instance
I20260811 19:03:31.000376 30146 external_mini_cluster.cc:1428] Setting key f92389f35f7befd23968f0fbbbb9f92d
I20260811 19:03:31.009797 2036 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P afab13f2822b4a56bdfb4093c6dad43b: Bootstrap starting.
I20260811 19:03:31.012851 2036 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P afab13f2822b4a56bdfb4093c6dad43b: Neither blocks nor log segments found. Creating new log.
I20260811 19:03:31.014137 2036 log.cc:824] T 00000000000000000000000000000000 P afab13f2822b4a56bdfb4093c6dad43b: Log is configured to *not* fsync() on all Append() calls
I20260811 19:03:31.015096 2045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:31.002497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.52:34773 (local address 127.0.0.1:33995)
0811 19:03:31.003518 (+ 1021us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:31.003535 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:31.003593 (+ 58us) server_negotiation.cc:407] Connection header received
0811 19:03:31.005458 (+ 1865us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:31.005480 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:31.005834 (+ 354us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:31.006178 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:31.012959 (+ 6781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:03:31.012972 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:03:31.012994 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:03:31.013025 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:03:31.013103 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:03:31.013129 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:03:31.013311 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:31.013490 (+ 179us) server_negotiation.cc:299] Negotiation successful
0811 19:03:31.013768 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":707,"thread_start_us":448,"threads_started":1}
I20260811 19:03:31.017321 2036 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P afab13f2822b4a56bdfb4093c6dad43b: No bootstrap required, opened a new log
I20260811 19:03:31.020396 30146 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 19:03:31.031309 2045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:31.027578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.52:45133 (local address 127.0.0.1:33995)
0811 19:03:31.027843 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:31.027851 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:31.028665 (+ 814us) server_negotiation.cc:407] Connection header received
0811 19:03:31.028852 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:31.028862 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:31.028969 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:31.029176 (+ 207us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:31.030340 (+ 1164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:03:31.030347 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:03:31.030352 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:03:31.030379 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:03:31.030454 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:03:31.030479 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:03:31.030705 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:31.030902 (+ 197us) server_negotiation.cc:299] Negotiation successful
0811 19:03:31.030983 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260811 19:03:31.031208 2036 raft_consensus.cc:348] T 00000000000000000000000000000000 P afab13f2822b4a56bdfb4093c6dad43b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afab13f2822b4a56bdfb4093c6dad43b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33995 } }
I20260811 19:03:31.031757 2036 raft_consensus.cc:374] T 00000000000000000000000000000000 P afab13f2822b4a56bdfb4093c6dad43b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:03:31.031867 2036 raft_consensus.cc:729] T 00000000000000000000000000000000 P afab13f2822b4a56bdfb4093c6dad43b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: afab13f2822b4a56bdfb4093c6dad43b, State: Initialized, Role: FOLLOWER
I20260811 19:03:31.032569 2036 consensus_queue.cc:260] T 00000000000000000000000000000000 P afab13f2822b4a56bdfb4093c6dad43b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afab13f2822b4a56bdfb4093c6dad43b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33995 } }
I20260811 19:03:31.032878 2036 raft_consensus.cc:388] T 00000000000000000000000000000000 P afab13f2822b4a56bdfb4093c6dad43b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:03:31.033049 2036 raft_consensus.cc:482] T 00000000000000000000000000000000 P afab13f2822b4a56bdfb4093c6dad43b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:03:31.033248 2036 raft_consensus.cc:3037] T 00000000000000000000000000000000 P afab13f2822b4a56bdfb4093c6dad43b [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:03:31.035696 2036 raft_consensus.cc:504] T 00000000000000000000000000000000 P afab13f2822b4a56bdfb4093c6dad43b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afab13f2822b4a56bdfb4093c6dad43b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33995 } }
I20260811 19:03:31.036247 2036 leader_election.cc:302] T 00000000000000000000000000000000 P afab13f2822b4a56bdfb4093c6dad43b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: afab13f2822b4a56bdfb4093c6dad43b; no voters:
I20260811 19:03:31.036944 2036 leader_election.cc:288] T 00000000000000000000000000000000 P afab13f2822b4a56bdfb4093c6dad43b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:03:31.037182 2058 raft_consensus.cc:2781] T 00000000000000000000000000000000 P afab13f2822b4a56bdfb4093c6dad43b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:03:31.037981 2058 raft_consensus.cc:686] T 00000000000000000000000000000000 P afab13f2822b4a56bdfb4093c6dad43b [term 1 LEADER]: Becoming Leader. State: Replica: afab13f2822b4a56bdfb4093c6dad43b, State: Running, Role: LEADER
I20260811 19:03:31.038658 2058 consensus_queue.cc:237] T 00000000000000000000000000000000 P afab13f2822b4a56bdfb4093c6dad43b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afab13f2822b4a56bdfb4093c6dad43b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33995 } }
I20260811 19:03:31.039897 2036 sys_catalog.cc:564] T 00000000000000000000000000000000 P afab13f2822b4a56bdfb4093c6dad43b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:03:31.042868 2059 sys_catalog.cc:455] T 00000000000000000000000000000000 P afab13f2822b4a56bdfb4093c6dad43b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "afab13f2822b4a56bdfb4093c6dad43b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afab13f2822b4a56bdfb4093c6dad43b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33995 } } }
I20260811 19:03:31.043623 2059 sys_catalog.cc:458] T 00000000000000000000000000000000 P afab13f2822b4a56bdfb4093c6dad43b [sys.catalog]: This master's current role is: LEADER
I20260811 19:03:31.043777 2060 sys_catalog.cc:455] T 00000000000000000000000000000000 P afab13f2822b4a56bdfb4093c6dad43b [sys.catalog]: SysCatalogTable state changed. Reason: New leader afab13f2822b4a56bdfb4093c6dad43b. Latest consensus state: current_term: 1 leader_uuid: "afab13f2822b4a56bdfb4093c6dad43b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afab13f2822b4a56bdfb4093c6dad43b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33995 } } }
I20260811 19:03:31.045272 2060 sys_catalog.cc:458] T 00000000000000000000000000000000 P afab13f2822b4a56bdfb4093c6dad43b [sys.catalog]: This master's current role is: LEADER
I20260811 19:03:31.056080 2073 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:03:31.062814 2073 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:03:31.074168 2073 catalog_manager.cc:1347] Generated new cluster ID: 6cd255bb5ac64412ab0d10b1b242e0bb
I20260811 19:03:31.074330 2073 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:03:31.094373 2073 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:03:31.094926 2073 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:03:31.102407 2073 catalog_manager.cc:5868] T 00000000000000000000000000000000 P afab13f2822b4a56bdfb4093c6dad43b: Generated new TSK 0
I20260811 19:03:31.103583 2073 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 19:03:31.111649 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 1920
W20260811 19:03:31.113150 2031 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.52:34773
W20260811 19:03:31.113920 30146 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 1920: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 19:03:31.114099 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 1920
2026-08-11T19:03:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (2433 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474615205622-30146-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:03:31 dist-test-slave-h6n0 krb5kdc[2096](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:03:31 dist-test-slave-h6n0 krb5kdc[2096](info): set up 2 sockets
Aug 11 19:03:31 dist-test-slave-h6n0 krb5kdc[2096](info): commencing operation
krb5kdc: starting...
W20260811 19:03:32.569993 30146 mini_kdc.cc:121] Time spent starting KDC: real 1.403s user 0.002s sys 0.051s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474615205622-30146-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474615205622-30146-0/krb5kdc/test-user.keytab.
Aug 11 19:03:32 dist-test-slave-h6n0 krb5kdc[2096](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475012, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:03:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:03:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.112.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab.
Entry for principal kudu/127.29.112.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab.
I20260811 19:03:32.926961 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474615205622-30146-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474615205622-30146-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474615205622-30146-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.190:45715
--webserver_interface=127.29.112.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab
--principal=kudu/127.29.112.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:36833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.190:45715
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:03:33.295540 2120 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:03:33.295853 2120 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:03:33.296051 2120 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:03:33.303876 2120 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:03:33.304013 2120 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:03:33.304103 2120 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:03:33.304210 2120 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:03:33.304275 2120 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:03:33.314666 2120 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:36833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474615205622-30146-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474615205622-30146-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.112.190:45715
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab
--principal=kudu/127.29.112.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.190:45715
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.112.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474615205622-30146-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:03:33.316099 2120 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:03:33.317488 2120 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:03:33.330699 2130 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:03:33.330668 2133 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:03:33.330915 2135 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:03:33.331523 2120 server_base.cc:1034] running on GCE node
I20260811 19:03:33.332350 2120 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:03:33.333798 2120 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:03:33.335067 2120 hybrid_clock.cc:648] HybridClock initialized: now 1786475013335021 us; error 53 us; skew 500 ppm
I20260811 19:03:33.335477 2120 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:03:33 dist-test-slave-h6n0 krb5kdc[2096](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475013, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:03:33.339196 2120 init.cc:377] Logged in from keytab as kudu/127.29.112.190@KRBTEST.COM (short username kudu)
I20260811 19:03:33.340965 2120 webserver.cc:460] Webserver started at http://127.29.112.190:46427/ using document root <none> and password file <none>
I20260811 19:03:33.341704 2120 fs_manager.cc:359] Metadata directory not provided
I20260811 19:03:33.341858 2120 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:03:33.342186 2120 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:03:33.345304 2120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474615205622-30146-0/minicluster-data/master-0/data/instance:
uuid: "442f693f8a824118a06f3f028060c558"
format_stamp: "Formatted at 2026-08-11 19:03:33 on dist-test-slave-h6n0"
server_key: "e48ee6d6cb4319a5e2b9c6137bfe3a5f"
server_key_iv: "228ffc322a91509bb4238a943476d0dd"
server_key_version: "encryptionkey@0"
I20260811 19:03:33.346082 2120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474615205622-30146-0/minicluster-data/master-0/wal/instance:
uuid: "442f693f8a824118a06f3f028060c558"
format_stamp: "Formatted at 2026-08-11 19:03:33 on dist-test-slave-h6n0"
server_key: "e48ee6d6cb4319a5e2b9c6137bfe3a5f"
server_key_iv: "228ffc322a91509bb4238a943476d0dd"
server_key_version: "encryptionkey@0"
I20260811 19:03:33.351583 2120 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260811 19:03:33.355443 2145 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:03:33.357061 2120 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 19:03:33.357270 2120 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474615205622-30146-0/minicluster-data/master-0/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474615205622-30146-0/minicluster-data/master-0/wal
uuid: "442f693f8a824118a06f3f028060c558"
format_stamp: "Formatted at 2026-08-11 19:03:33 on dist-test-slave-h6n0"
server_key: "e48ee6d6cb4319a5e2b9c6137bfe3a5f"
server_key_iv: "228ffc322a91509bb4238a943476d0dd"
server_key_version: "encryptionkey@0"
I20260811 19:03:33.357493 2120 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474615205622-30146-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474615205622-30146-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474615205622-30146-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:03:33.372509 2120 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:03:33.377444 2120 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:03:33.377760 2120 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:03:33.398379 2120 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.190:45715
I20260811 19:03:33.398649 2228 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.190:45715 every 8 connection(s)
I20260811 19:03:33.399909 2120 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
I20260811 19:03:33.402601 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 2120
I20260811 19:03:33.402802 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474615205622-30146-0/minicluster-data/master-0/wal/instance
I20260811 19:03:33.403266 30146 external_mini_cluster.cc:1428] Setting key cea4ccfce169338fc893ec3951d41075
I20260811 19:03:33.403959 2229 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:03:33.418797 2229 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 442f693f8a824118a06f3f028060c558: Bootstrap starting.
Aug 11 19:03:33 dist-test-slave-h6n0 krb5kdc[2096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475012, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 19:03:33.424487 2229 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 442f693f8a824118a06f3f028060c558: Neither blocks nor log segments found. Creating new log.
I20260811 19:03:33.426368 2229 log.cc:824] T 00000000000000000000000000000000 P 442f693f8a824118a06f3f028060c558: Log is configured to *not* fsync() on all Append() calls
I20260811 19:03:33.430968 2229 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 442f693f8a824118a06f3f028060c558: No bootstrap required, opened a new log
I20260811 19:03:33.432533 2233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:33.405205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36743 (local address 127.29.112.190:45715)
0811 19:03:33.406020 (+ 815us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:33.406036 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:33.406085 (+ 49us) server_negotiation.cc:407] Connection header received
0811 19:03:33.408732 (+ 2647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:33.408759 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:33.409248 (+ 489us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:33.409686 (+ 438us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:33.411652 (+ 1966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:33.413509 (+ 1857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:33.414952 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:33.416105 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:33.416311 (+ 206us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:33.421302 (+ 4991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:03:33.421325 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:03:33.421354 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:03:33.421441 (+ 87us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:03:33.426730 (+ 5289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:33.428335 (+ 1605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:33.428343 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:33.428350 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:33.428637 (+ 287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:33.429141 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:33.429149 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:33.429156 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:33.429926 (+ 770us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:03:33.430313 (+ 387us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:33.430755 (+ 442us) server_negotiation.cc:299] Negotiation successful
0811 19:03:33.431140 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":589,"thread_start_us":371,"threads_started":1}
I20260811 19:03:33.437108 30146 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 19:03:33.440505 2229 raft_consensus.cc:348] T 00000000000000000000000000000000 P 442f693f8a824118a06f3f028060c558 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "442f693f8a824118a06f3f028060c558" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 45715 } }
I20260811 19:03:33.440896 2229 raft_consensus.cc:374] T 00000000000000000000000000000000 P 442f693f8a824118a06f3f028060c558 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:03:33.441021 2229 raft_consensus.cc:729] T 00000000000000000000000000000000 P 442f693f8a824118a06f3f028060c558 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 442f693f8a824118a06f3f028060c558, State: Initialized, Role: FOLLOWER
I20260811 19:03:33.441658 2229 consensus_queue.cc:260] T 00000000000000000000000000000000 P 442f693f8a824118a06f3f028060c558 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "442f693f8a824118a06f3f028060c558" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 45715 } }
I20260811 19:03:33.441916 2229 raft_consensus.cc:388] T 00000000000000000000000000000000 P 442f693f8a824118a06f3f028060c558 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:03:33.442071 2229 raft_consensus.cc:482] T 00000000000000000000000000000000 P 442f693f8a824118a06f3f028060c558 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:03:33.442248 2229 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 442f693f8a824118a06f3f028060c558 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:03:33.445801 2229 raft_consensus.cc:504] T 00000000000000000000000000000000 P 442f693f8a824118a06f3f028060c558 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "442f693f8a824118a06f3f028060c558" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 45715 } }
I20260811 19:03:33.446385 2229 leader_election.cc:302] T 00000000000000000000000000000000 P 442f693f8a824118a06f3f028060c558 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 442f693f8a824118a06f3f028060c558; no voters:
I20260811 19:03:33.447118 2229 leader_election.cc:288] T 00000000000000000000000000000000 P 442f693f8a824118a06f3f028060c558 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:03:33.447240 2246 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 442f693f8a824118a06f3f028060c558 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:03:33.449014 2246 raft_consensus.cc:686] T 00000000000000000000000000000000 P 442f693f8a824118a06f3f028060c558 [term 1 LEADER]: Becoming Leader. State: Replica: 442f693f8a824118a06f3f028060c558, State: Running, Role: LEADER
I20260811 19:03:33.449749 2246 consensus_queue.cc:237] T 00000000000000000000000000000000 P 442f693f8a824118a06f3f028060c558 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "442f693f8a824118a06f3f028060c558" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 45715 } }
I20260811 19:03:33.450014 2229 sys_catalog.cc:564] T 00000000000000000000000000000000 P 442f693f8a824118a06f3f028060c558 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:03:33.453886 2254 sys_catalog.cc:455] T 00000000000000000000000000000000 P 442f693f8a824118a06f3f028060c558 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 442f693f8a824118a06f3f028060c558. Latest consensus state: current_term: 1 leader_uuid: "442f693f8a824118a06f3f028060c558" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "442f693f8a824118a06f3f028060c558" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 45715 } } }
I20260811 19:03:33.454474 2254 sys_catalog.cc:458] T 00000000000000000000000000000000 P 442f693f8a824118a06f3f028060c558 [sys.catalog]: This master's current role is: LEADER
I20260811 19:03:33.463541 2233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:33.442723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57519 (local address 127.29.112.190:45715)
0811 19:03:33.442960 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:33.442973 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:33.443476 (+ 503us) server_negotiation.cc:407] Connection header received
0811 19:03:33.443689 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:33.443693 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:33.443760 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:33.443854 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:33.445950 (+ 2096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:33.446947 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:33.448763 (+ 1816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:33.449625 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:33.449821 (+ 196us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:33.452889 (+ 3068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:03:33.452897 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:03:33.452902 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:03:33.452947 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:03:33.457833 (+ 4886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:33.459291 (+ 1458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:33.459300 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:33.459306 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:33.459505 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:33.460251 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:33.460259 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:33.460264 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:33.460790 (+ 526us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:03:33.462465 (+ 1675us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:33.462744 (+ 279us) server_negotiation.cc:299] Negotiation successful
0811 19:03:33.462994 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 19:03:33.467976 2250 sys_catalog.cc:455] T 00000000000000000000000000000000 P 442f693f8a824118a06f3f028060c558 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "442f693f8a824118a06f3f028060c558" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "442f693f8a824118a06f3f028060c558" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 45715 } } }
I20260811 19:03:33.468333 2250 sys_catalog.cc:458] T 00000000000000000000000000000000 P 442f693f8a824118a06f3f028060c558 [sys.catalog]: This master's current role is: LEADER
I20260811 19:03:33.469570 2265 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:03:33.477672 2265 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:03:33.486680 2265 catalog_manager.cc:1347] Generated new cluster ID: 8028bed092164a13bb71c73bc51e40ba
I20260811 19:03:33.486799 2265 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:03:33.504891 2265 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:03:33.506855 2265 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:03:33.513813 2265 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 442f693f8a824118a06f3f028060c558: Generated new TSK 0
I20260811 19:03:33.514829 2265 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 19:03:33.524789 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 2120
I20260811 19:03:33.526036 2222 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:03:33.620687 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 2120
W20260811 19:03:33.647450 2093 connection.cc:535] client connection to 127.29.112.190:45715 recv error: Network error: failed to read from TLS socket (remote: 127.29.112.190:45715): Connection reset by peer (error 104)
2026-08-11T19:03:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2500 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474615205622-30146-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:03:33 dist-test-slave-h6n0 krb5kdc[2290](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:03:33 dist-test-slave-h6n0 krb5kdc[2290](info): set up 2 sockets
Aug 11 19:03:33 dist-test-slave-h6n0 krb5kdc[2290](info): commencing operation
krb5kdc: starting...
W20260811 19:03:35.076263 30146 mini_kdc.cc:121] Time spent starting KDC: real 1.409s user 0.007s sys 0.051s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474615205622-30146-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474615205622-30146-0/krb5kdc/test-user.keytab.
Aug 11 19:03:35 dist-test-slave-h6n0 krb5kdc[2290](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475015, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:03:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:03:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.112.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab.
Entry for principal kudu/127.29.112.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab.
I20260811 19:03:35.444417 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474615205622-30146-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474615205622-30146-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474615205622-30146-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.190:41505
--webserver_interface=127.29.112.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab
--principal=kudu/127.29.112.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:43247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.190:41505
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:03:35.811762 2315 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:03:35.812053 2315 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:03:35.812222 2315 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:03:35.819516 2315 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:03:35.819625 2315 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:03:35.819712 2315 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:03:35.819831 2315 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:03:35.819895 2315 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:03:35.831676 2315 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:43247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474615205622-30146-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474615205622-30146-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.112.190:41505
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab
--principal=kudu/127.29.112.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.190:41505
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.112.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474615205622-30146-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:03:35.833142 2315 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:03:35.834597 2315 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:03:35.847709 2321 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:03:35.847968 2322 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:03:35.848608 2332 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:03:35.849422 2315 server_base.cc:1034] running on GCE node
I20260811 19:03:35.850373 2315 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:03:35.851902 2315 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:03:35.853163 2315 hybrid_clock.cc:648] HybridClock initialized: now 1786475015853124 us; error 40 us; skew 500 ppm
I20260811 19:03:35.853552 2315 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:03:35 dist-test-slave-h6n0 krb5kdc[2290](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475015, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:03:35.857597 2315 init.cc:377] Logged in from keytab as kudu/127.29.112.190@KRBTEST.COM (short username kudu)
I20260811 19:03:35.859649 2315 webserver.cc:460] Webserver started at http://127.29.112.190:43735/ using document root <none> and password file <none>
I20260811 19:03:35.860404 2315 fs_manager.cc:359] Metadata directory not provided
I20260811 19:03:35.860577 2315 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:03:35.860924 2315 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:03:35.864030 2315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474615205622-30146-0/minicluster-data/master-0/data/instance:
uuid: "83b2456166584af69cd13fade5007ef1"
format_stamp: "Formatted at 2026-08-11 19:03:35 on dist-test-slave-h6n0"
server_key: "fbbbc63b3d11f0197b8b141f9c728f3d"
server_key_iv: "3fd5afa2e7e339e1e5de3ebf6c994ce0"
server_key_version: "encryptionkey@0"
I20260811 19:03:35.864828 2315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474615205622-30146-0/minicluster-data/master-0/wal/instance:
uuid: "83b2456166584af69cd13fade5007ef1"
format_stamp: "Formatted at 2026-08-11 19:03:35 on dist-test-slave-h6n0"
server_key: "fbbbc63b3d11f0197b8b141f9c728f3d"
server_key_iv: "3fd5afa2e7e339e1e5de3ebf6c994ce0"
server_key_version: "encryptionkey@0"
I20260811 19:03:35.870829 2315 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.003s
I20260811 19:03:35.874722 2347 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:03:35.876164 2315 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 19:03:35.876382 2315 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474615205622-30146-0/minicluster-data/master-0/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474615205622-30146-0/minicluster-data/master-0/wal
uuid: "83b2456166584af69cd13fade5007ef1"
format_stamp: "Formatted at 2026-08-11 19:03:35 on dist-test-slave-h6n0"
server_key: "fbbbc63b3d11f0197b8b141f9c728f3d"
server_key_iv: "3fd5afa2e7e339e1e5de3ebf6c994ce0"
server_key_version: "encryptionkey@0"
I20260811 19:03:35.876624 2315 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474615205622-30146-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474615205622-30146-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474615205622-30146-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:03:35.894136 2315 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:03:35.900451 2315 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:03:35.900781 2315 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:03:35.921557 2315 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.190:41505
I20260811 19:03:35.921593 2441 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.190:41505 every 8 connection(s)
I20260811 19:03:35.923079 2315 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
I20260811 19:03:35.928332 2443 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:03:35.929919 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 2315
I20260811 19:03:35.930117 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474615205622-30146-0/minicluster-data/master-0/wal/instance
I20260811 19:03:35.930769 30146 external_mini_cluster.cc:1428] Setting key d191ec11173bda3351a13e35b658a517
I20260811 19:03:35.943691 2443 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83b2456166584af69cd13fade5007ef1: Bootstrap starting.
I20260811 19:03:35.946362 2451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:35.932706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51273 (local address 127.29.112.190:41505)
0811 19:03:35.933646 (+ 940us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:35.933659 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:35.933707 (+ 48us) server_negotiation.cc:407] Connection header received
0811 19:03:35.936094 (+ 2387us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:35.936140 (+ 46us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:35.936688 (+ 548us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:35.937063 (+ 375us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:35.937737 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:35.939337 (+ 1600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:35.941337 (+ 2000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:35.942105 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:35.942262 (+ 157us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:35.943726 (+ 1464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:03:35.943735 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:03:35.943748 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:03:35.944423 (+ 675us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:03:35.944529 (+ 106us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:03:35.944559 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:03:35.944703 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:35.945086 (+ 383us) server_negotiation.cc:299] Negotiation successful
0811 19:03:35.945328 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":701,"thread_start_us":460,"threads_started":1}
I20260811 19:03:35.947124 2443 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 83b2456166584af69cd13fade5007ef1: Neither blocks nor log segments found. Creating new log.
I20260811 19:03:35.948221 2443 log.cc:824] T 00000000000000000000000000000000 P 83b2456166584af69cd13fade5007ef1: Log is configured to *not* fsync() on all Append() calls
I20260811 19:03:35.951293 30146 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 19:03:35.952165 2443 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83b2456166584af69cd13fade5007ef1: No bootstrap required, opened a new log
I20260811 19:03:35.965762 2443 raft_consensus.cc:348] T 00000000000000000000000000000000 P 83b2456166584af69cd13fade5007ef1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83b2456166584af69cd13fade5007ef1" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 41505 } }
I20260811 19:03:35.966276 2443 raft_consensus.cc:374] T 00000000000000000000000000000000 P 83b2456166584af69cd13fade5007ef1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:03:35.966384 2443 raft_consensus.cc:729] T 00000000000000000000000000000000 P 83b2456166584af69cd13fade5007ef1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83b2456166584af69cd13fade5007ef1, State: Initialized, Role: FOLLOWER
I20260811 19:03:35.967139 2443 consensus_queue.cc:260] T 00000000000000000000000000000000 P 83b2456166584af69cd13fade5007ef1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83b2456166584af69cd13fade5007ef1" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 41505 } }
I20260811 19:03:35.967473 2443 raft_consensus.cc:388] T 00000000000000000000000000000000 P 83b2456166584af69cd13fade5007ef1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:03:35.967644 2443 raft_consensus.cc:482] T 00000000000000000000000000000000 P 83b2456166584af69cd13fade5007ef1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:03:35.967859 2443 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 83b2456166584af69cd13fade5007ef1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:03:35.971923 2443 raft_consensus.cc:504] T 00000000000000000000000000000000 P 83b2456166584af69cd13fade5007ef1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83b2456166584af69cd13fade5007ef1" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 41505 } }
I20260811 19:03:35.972514 2443 leader_election.cc:302] T 00000000000000000000000000000000 P 83b2456166584af69cd13fade5007ef1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 83b2456166584af69cd13fade5007ef1; no voters:
I20260811 19:03:35.973311 2443 leader_election.cc:288] T 00000000000000000000000000000000 P 83b2456166584af69cd13fade5007ef1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:03:35.973464 2467 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 83b2456166584af69cd13fade5007ef1 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:03:35.977044 2443 sys_catalog.cc:564] T 00000000000000000000000000000000 P 83b2456166584af69cd13fade5007ef1 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:03:35.977415 2451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:35.968378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42617 (local address 127.29.112.190:41505)
0811 19:03:35.968580 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:35.968585 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:35.969188 (+ 603us) server_negotiation.cc:407] Connection header received
0811 19:03:35.969446 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:35.969454 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:35.969554 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:35.969684 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:35.970385 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:35.971462 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:35.973185 (+ 1723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:35.974261 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:35.975245 (+ 984us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:35.976172 (+ 927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:03:35.976180 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:03:35.976184 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:03:35.976222 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:03:35.976370 (+ 148us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:03:35.976394 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:03:35.976778 (+ 384us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:35.976937 (+ 159us) server_negotiation.cc:299] Negotiation successful
0811 19:03:35.977004 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 19:03:35.979031 2467 raft_consensus.cc:686] T 00000000000000000000000000000000 P 83b2456166584af69cd13fade5007ef1 [term 1 LEADER]: Becoming Leader. State: Replica: 83b2456166584af69cd13fade5007ef1, State: Running, Role: LEADER
I20260811 19:03:35.979871 2467 consensus_queue.cc:237] T 00000000000000000000000000000000 P 83b2456166584af69cd13fade5007ef1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83b2456166584af69cd13fade5007ef1" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 41505 } }
I20260811 19:03:35.984747 2468 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83b2456166584af69cd13fade5007ef1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "83b2456166584af69cd13fade5007ef1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83b2456166584af69cd13fade5007ef1" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 41505 } } }
I20260811 19:03:35.984680 2470 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83b2456166584af69cd13fade5007ef1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 83b2456166584af69cd13fade5007ef1. Latest consensus state: current_term: 1 leader_uuid: "83b2456166584af69cd13fade5007ef1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83b2456166584af69cd13fade5007ef1" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 41505 } } }
I20260811 19:03:35.985225 2470 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83b2456166584af69cd13fade5007ef1 [sys.catalog]: This master's current role is: LEADER
I20260811 19:03:35.985224 2468 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83b2456166584af69cd13fade5007ef1 [sys.catalog]: This master's current role is: LEADER
W20260811 19:03:35.999943 2492 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 83b2456166584af69cd13fade5007ef1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 19:03:36.000121 2492 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 19:03:36.000270 2477 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:03:36.002506 2477 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:03:36.011451 2477 catalog_manager.cc:1347] Generated new cluster ID: 79f3cda4895840b9be43df6f47e52f76
I20260811 19:03:36.011569 2477 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:03:36.029006 2477 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:03:36.030882 2477 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:03:36.045406 2477 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 83b2456166584af69cd13fade5007ef1: Generated new TSK 0
I20260811 19:03:36.046489 2477 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 19:03:36.066610 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 2315
W20260811 19:03:36.068419 2432 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:51273
W20260811 19:03:36.069574 30146 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 2315: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 19:03:36.069891 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 2315
W20260811 19:03:36.096498 2288 connection.cc:535] client connection to 127.29.112.190:41505 recv error: Network error: failed to read from TLS socket (remote: 127.29.112.190:41505): Connection reset by peer (error 104)
2026-08-11T19:03:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (2448 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474615205622-30146-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:03:36 dist-test-slave-h6n0 krb5kdc[2507](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:03:36 dist-test-slave-h6n0 krb5kdc[2507](info): set up 2 sockets
Aug 11 19:03:36 dist-test-slave-h6n0 krb5kdc[2507](info): commencing operation
krb5kdc: starting...
W20260811 19:03:37.514587 30146 mini_kdc.cc:121] Time spent starting KDC: real 1.399s user 0.000s sys 0.053s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474615205622-30146-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474615205622-30146-0/krb5kdc/test-user.keytab.
Aug 11 19:03:37 dist-test-slave-h6n0 krb5kdc[2507](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475017, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:03:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:03:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.112.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab.
Entry for principal kudu/127.29.112.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab.
I20260811 19:03:37.877921 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474615205622-30146-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474615205622-30146-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474615205622-30146-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.190:41225
--webserver_interface=127.29.112.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab
--principal=kudu/127.29.112.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:37949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.190:41225
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:03:38.243758 2526 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:03:38.244086 2526 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:03:38.244271 2526 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:03:38.252522 2526 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:03:38.252728 2526 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:03:38.253334 2526 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:03:38.253501 2526 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:03:38.253592 2526 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:03:38.265851 2526 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:37949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474615205622-30146-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474615205622-30146-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.112.190:41225
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab
--principal=kudu/127.29.112.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.190:41225
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.112.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474615205622-30146-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:03:38.267318 2526 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:03:38.268795 2526 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:03:38.289908 2534 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:03:38.290696 2533 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:03:38.291085 2539 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:03:38.291623 2526 server_base.cc:1034] running on GCE node
I20260811 19:03:38.292788 2526 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:03:38.294680 2526 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:03:38.296098 2526 hybrid_clock.cc:648] HybridClock initialized: now 1786475018295967 us; error 148 us; skew 500 ppm
I20260811 19:03:38.296496 2526 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:03:38 dist-test-slave-h6n0 krb5kdc[2507](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475018, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:03:38.300275 2526 init.cc:377] Logged in from keytab as kudu/127.29.112.190@KRBTEST.COM (short username kudu)
I20260811 19:03:38.302102 2526 webserver.cc:460] Webserver started at http://127.29.112.190:45725/ using document root <none> and password file <none>
I20260811 19:03:38.302947 2526 fs_manager.cc:359] Metadata directory not provided
I20260811 19:03:38.303103 2526 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:03:38.303437 2526 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:03:38.307082 2526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474615205622-30146-0/minicluster-data/master-0/data/instance:
uuid: "0e830aca9d2b4ec991a9fc3f5eef2378"
format_stamp: "Formatted at 2026-08-11 19:03:38 on dist-test-slave-h6n0"
server_key: "cc0ee1ea2ea1205575d5fc2faf571c67"
server_key_iv: "70ccac7c3f83504855c01329d70b9375"
server_key_version: "encryptionkey@0"
I20260811 19:03:38.307928 2526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474615205622-30146-0/minicluster-data/master-0/wal/instance:
uuid: "0e830aca9d2b4ec991a9fc3f5eef2378"
format_stamp: "Formatted at 2026-08-11 19:03:38 on dist-test-slave-h6n0"
server_key: "cc0ee1ea2ea1205575d5fc2faf571c67"
server_key_iv: "70ccac7c3f83504855c01329d70b9375"
server_key_version: "encryptionkey@0"
I20260811 19:03:38.314766 2526 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260811 19:03:38.319968 2550 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:03:38.321833 2526 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.003s
I20260811 19:03:38.322049 2526 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474615205622-30146-0/minicluster-data/master-0/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474615205622-30146-0/minicluster-data/master-0/wal
uuid: "0e830aca9d2b4ec991a9fc3f5eef2378"
format_stamp: "Formatted at 2026-08-11 19:03:38 on dist-test-slave-h6n0"
server_key: "cc0ee1ea2ea1205575d5fc2faf571c67"
server_key_iv: "70ccac7c3f83504855c01329d70b9375"
server_key_version: "encryptionkey@0"
I20260811 19:03:38.322629 2526 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474615205622-30146-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474615205622-30146-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474615205622-30146-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:03:38.331476 2526 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:03:38.338064 2526 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:03:38.338445 2526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:03:38.362399 2526 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.190:41225
I20260811 19:03:38.362453 2634 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.190:41225 every 8 connection(s)
I20260811 19:03:38.363973 2526 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
I20260811 19:03:38.368827 2635 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:03:38.374253 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 2526
I20260811 19:03:38.374490 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474615205622-30146-0/minicluster-data/master-0/wal/instance
I20260811 19:03:38.375108 30146 external_mini_cluster.cc:1428] Setting key e624cbc0048b0a7f5fffd605857d364d
I20260811 19:03:38.385026 2635 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e830aca9d2b4ec991a9fc3f5eef2378: Bootstrap starting.
I20260811 19:03:38.387022 2639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:38.378120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56229 (local address 127.29.112.190:41225)
0811 19:03:38.379317 (+ 1197us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:38.379333 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:38.379514 (+ 181us) server_negotiation.cc:407] Connection header received
0811 19:03:38.382737 (+ 3223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:38.382779 (+ 42us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:38.383188 (+ 409us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:38.383663 (+ 475us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:38.385113 (+ 1450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:03:38.385120 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:03:38.385134 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:03:38.385150 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:03:38.385221 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:03:38.385279 (+ 58us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:03:38.385535 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:38.385677 (+ 142us) server_negotiation.cc:299] Negotiation successful
0811 19:03:38.385906 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":870,"thread_start_us":568,"threads_started":1}
I20260811 19:03:38.390199 2635 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0e830aca9d2b4ec991a9fc3f5eef2378: Neither blocks nor log segments found. Creating new log.
I20260811 19:03:38.391676 2635 log.cc:824] T 00000000000000000000000000000000 P 0e830aca9d2b4ec991a9fc3f5eef2378: Log is configured to *not* fsync() on all Append() calls
I20260811 19:03:38.393194 30146 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 19:03:38.396042 2635 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e830aca9d2b4ec991a9fc3f5eef2378: No bootstrap required, opened a new log
I20260811 19:03:38.404072 2639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:38.399987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49265 (local address 127.29.112.190:41225)
0811 19:03:38.400465 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:38.400483 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:38.401186 (+ 703us) server_negotiation.cc:407] Connection header received
0811 19:03:38.401394 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:38.401398 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:38.401455 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:38.401555 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:38.402975 (+ 1420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:03:38.402982 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:03:38.402986 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:03:38.403014 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:03:38.403103 (+ 89us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:03:38.403138 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:03:38.403439 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:38.403568 (+ 129us) server_negotiation.cc:299] Negotiation successful
0811 19:03:38.403641 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260811 19:03:38.407225 2635 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0e830aca9d2b4ec991a9fc3f5eef2378 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e830aca9d2b4ec991a9fc3f5eef2378" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 41225 } }
I20260811 19:03:38.407655 2635 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0e830aca9d2b4ec991a9fc3f5eef2378 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:03:38.407845 2635 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0e830aca9d2b4ec991a9fc3f5eef2378 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e830aca9d2b4ec991a9fc3f5eef2378, State: Initialized, Role: FOLLOWER
I20260811 19:03:38.408563 2635 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0e830aca9d2b4ec991a9fc3f5eef2378 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e830aca9d2b4ec991a9fc3f5eef2378" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 41225 } }
I20260811 19:03:38.408800 2635 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0e830aca9d2b4ec991a9fc3f5eef2378 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:03:38.409003 2635 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0e830aca9d2b4ec991a9fc3f5eef2378 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:03:38.409178 2635 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0e830aca9d2b4ec991a9fc3f5eef2378 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:03:38.412467 2635 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0e830aca9d2b4ec991a9fc3f5eef2378 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e830aca9d2b4ec991a9fc3f5eef2378" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 41225 } }
I20260811 19:03:38.413071 2635 leader_election.cc:302] T 00000000000000000000000000000000 P 0e830aca9d2b4ec991a9fc3f5eef2378 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0e830aca9d2b4ec991a9fc3f5eef2378; no voters:
I20260811 19:03:38.413918 2635 leader_election.cc:288] T 00000000000000000000000000000000 P 0e830aca9d2b4ec991a9fc3f5eef2378 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:03:38.413964 2655 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0e830aca9d2b4ec991a9fc3f5eef2378 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:03:38.415016 2655 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0e830aca9d2b4ec991a9fc3f5eef2378 [term 1 LEADER]: Becoming Leader. State: Replica: 0e830aca9d2b4ec991a9fc3f5eef2378, State: Running, Role: LEADER
I20260811 19:03:38.415879 2655 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0e830aca9d2b4ec991a9fc3f5eef2378 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e830aca9d2b4ec991a9fc3f5eef2378" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 41225 } }
I20260811 19:03:38.417353 2635 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0e830aca9d2b4ec991a9fc3f5eef2378 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:03:38.421402 2659 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e830aca9d2b4ec991a9fc3f5eef2378 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0e830aca9d2b4ec991a9fc3f5eef2378. Latest consensus state: current_term: 1 leader_uuid: "0e830aca9d2b4ec991a9fc3f5eef2378" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e830aca9d2b4ec991a9fc3f5eef2378" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 41225 } } }
I20260811 19:03:38.421482 2660 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e830aca9d2b4ec991a9fc3f5eef2378 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0e830aca9d2b4ec991a9fc3f5eef2378" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e830aca9d2b4ec991a9fc3f5eef2378" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 41225 } } }
I20260811 19:03:38.422461 2660 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e830aca9d2b4ec991a9fc3f5eef2378 [sys.catalog]: This master's current role is: LEADER
I20260811 19:03:38.422459 2659 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e830aca9d2b4ec991a9fc3f5eef2378 [sys.catalog]: This master's current role is: LEADER
I20260811 19:03:38.431294 2668 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:03:38.441082 2668 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:03:38.455397 2668 catalog_manager.cc:1347] Generated new cluster ID: f395ced47d1d413d939934346c27c214
I20260811 19:03:38.455561 2668 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:03:38.475045 2668 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:03:38.475663 2668 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:03:38.492298 2668 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0e830aca9d2b4ec991a9fc3f5eef2378: Generated new TSK 0
I20260811 19:03:38.493257 2668 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 19:03:38.511076 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 2526
W20260811 19:03:38.513006 2627 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:56229
W20260811 19:03:38.514014 30146 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 2526: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 19:03:38.514217 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 2526
2026-08-11T19:03:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (2448 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474615205622-30146-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:03:38 dist-test-slave-h6n0 krb5kdc[2690](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:03:38 dist-test-slave-h6n0 krb5kdc[2690](info): set up 2 sockets
Aug 11 19:03:38 dist-test-slave-h6n0 krb5kdc[2690](info): commencing operation
krb5kdc: starting...
W20260811 19:03:39.968665 30146 mini_kdc.cc:121] Time spent starting KDC: real 1.403s user 0.001s sys 0.051s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474615205622-30146-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474615205622-30146-0/krb5kdc/test-user.keytab.
Aug 11 19:03:40 dist-test-slave-h6n0 krb5kdc[2690](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475020, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:03:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:03:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.112.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab.
Entry for principal kudu/127.29.112.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab.
I20260811 19:03:40.330399 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474615205622-30146-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474615205622-30146-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474615205622-30146-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.190:39809
--webserver_interface=127.29.112.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab
--principal=kudu/127.29.112.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:40581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.190:39809
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:03:40.721726 2710 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:03:40.722111 2710 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:03:40.722297 2710 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:03:40.730041 2710 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:03:40.730226 2710 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:03:40.730347 2710 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:03:40.730486 2710 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:03:40.730614 2710 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:03:40.741014 2710 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:40581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474615205622-30146-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474615205622-30146-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.112.190:39809
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab
--principal=kudu/127.29.112.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.190:39809
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.112.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474615205622-30146-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:03:40.742455 2710 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:03:40.743891 2710 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:03:40.757694 2718 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:03:40.757664 2720 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:03:40.757875 2723 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:03:40.759604 2710 server_base.cc:1034] running on GCE node
I20260811 19:03:40.760502 2710 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:03:40.762023 2710 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:03:40.763329 2710 hybrid_clock.cc:648] HybridClock initialized: now 1786475020763305 us; error 47 us; skew 500 ppm
I20260811 19:03:40.763726 2710 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:03:40 dist-test-slave-h6n0 krb5kdc[2690](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475020, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:03:40.767412 2710 init.cc:377] Logged in from keytab as kudu/127.29.112.190@KRBTEST.COM (short username kudu)
I20260811 19:03:40.769086 2710 webserver.cc:460] Webserver started at http://127.29.112.190:44719/ using document root <none> and password file <none>
I20260811 19:03:40.769824 2710 fs_manager.cc:359] Metadata directory not provided
I20260811 19:03:40.770000 2710 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:03:40.770334 2710 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:03:40.774950 2710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474615205622-30146-0/minicluster-data/master-0/data/instance:
uuid: "f5029f037aa44a50994330e0f70b9bf2"
format_stamp: "Formatted at 2026-08-11 19:03:40 on dist-test-slave-h6n0"
server_key: "230bc7bdb2b6175d8e3e6902f1dc8a63"
server_key_iv: "3d3138a1c39c8d9607ed51b4d0cf5d17"
server_key_version: "encryptionkey@0"
I20260811 19:03:40.776002 2710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474615205622-30146-0/minicluster-data/master-0/wal/instance:
uuid: "f5029f037aa44a50994330e0f70b9bf2"
format_stamp: "Formatted at 2026-08-11 19:03:40 on dist-test-slave-h6n0"
server_key: "230bc7bdb2b6175d8e3e6902f1dc8a63"
server_key_iv: "3d3138a1c39c8d9607ed51b4d0cf5d17"
server_key_version: "encryptionkey@0"
I20260811 19:03:40.782806 2710 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260811 19:03:40.787528 2735 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:03:40.789338 2710 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 19:03:40.789574 2710 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474615205622-30146-0/minicluster-data/master-0/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474615205622-30146-0/minicluster-data/master-0/wal
uuid: "f5029f037aa44a50994330e0f70b9bf2"
format_stamp: "Formatted at 2026-08-11 19:03:40 on dist-test-slave-h6n0"
server_key: "230bc7bdb2b6175d8e3e6902f1dc8a63"
server_key_iv: "3d3138a1c39c8d9607ed51b4d0cf5d17"
server_key_version: "encryptionkey@0"
I20260811 19:03:40.789812 2710 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474615205622-30146-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474615205622-30146-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474615205622-30146-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:03:40.821054 2710 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:03:40.825425 2710 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:03:40.825762 2710 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:03:40.847592 2710 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.190:39809
I20260811 19:03:40.847615 2835 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.190:39809 every 8 connection(s)
I20260811 19:03:40.849887 2710 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
I20260811 19:03:40.854194 2836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:03:40.858718 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 2710
I20260811 19:03:40.858933 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474615205622-30146-0/minicluster-data/master-0/wal/instance
I20260811 19:03:40.859514 30146 external_mini_cluster.cc:1428] Setting key 0921ed97989c3d77a4144328dbf6a049
I20260811 19:03:40.866375 2836 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5029f037aa44a50994330e0f70b9bf2: Bootstrap starting.
I20260811 19:03:40.870054 2836 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f5029f037aa44a50994330e0f70b9bf2: Neither blocks nor log segments found. Creating new log.
I20260811 19:03:40.871261 2836 log.cc:824] T 00000000000000000000000000000000 P f5029f037aa44a50994330e0f70b9bf2: Log is configured to *not* fsync() on all Append() calls
I20260811 19:03:40.874847 2836 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5029f037aa44a50994330e0f70b9bf2: No bootstrap required, opened a new log
Aug 11 19:03:40 dist-test-slave-h6n0 krb5kdc[2690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475020, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 19:03:40.883157 2836 raft_consensus.cc:348] T 00000000000000000000000000000000 P f5029f037aa44a50994330e0f70b9bf2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5029f037aa44a50994330e0f70b9bf2" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 39809 } }
I20260811 19:03:40.883576 2836 raft_consensus.cc:374] T 00000000000000000000000000000000 P f5029f037aa44a50994330e0f70b9bf2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:03:40.883718 2836 raft_consensus.cc:729] T 00000000000000000000000000000000 P f5029f037aa44a50994330e0f70b9bf2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5029f037aa44a50994330e0f70b9bf2, State: Initialized, Role: FOLLOWER
I20260811 19:03:40.884402 2836 consensus_queue.cc:260] T 00000000000000000000000000000000 P f5029f037aa44a50994330e0f70b9bf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5029f037aa44a50994330e0f70b9bf2" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 39809 } }
I20260811 19:03:40.884745 2836 raft_consensus.cc:388] T 00000000000000000000000000000000 P f5029f037aa44a50994330e0f70b9bf2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:03:40.884909 2836 raft_consensus.cc:482] T 00000000000000000000000000000000 P f5029f037aa44a50994330e0f70b9bf2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:03:40.885111 2836 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f5029f037aa44a50994330e0f70b9bf2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:03:40.885947 2841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:40.861409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57343 (local address 127.29.112.190:39809)
0811 19:03:40.862259 (+ 850us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:40.862270 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:40.862314 (+ 44us) server_negotiation.cc:407] Connection header received
0811 19:03:40.864834 (+ 2520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:40.864864 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:40.865296 (+ 432us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:40.865614 (+ 318us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:40.867548 (+ 1934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:40.868909 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:40.870791 (+ 1882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:40.871533 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:40.871675 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:40.876950 (+ 5275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:03:40.876962 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:03:40.876975 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:03:40.877016 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:03:40.880999 (+ 3983us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:40.882011 (+ 1012us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:40.882019 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:40.882025 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:40.882273 (+ 248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:40.882798 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:40.882802 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:40.882804 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:40.883370 (+ 566us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:03:40.883725 (+ 355us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:40.884201 (+ 476us) server_negotiation.cc:299] Negotiation successful
0811 19:03:40.884645 (+ 444us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":642,"thread_start_us":392,"threads_started":1}
I20260811 19:03:40.888162 2836 raft_consensus.cc:504] T 00000000000000000000000000000000 P f5029f037aa44a50994330e0f70b9bf2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5029f037aa44a50994330e0f70b9bf2" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 39809 } }
I20260811 19:03:40.888756 2836 leader_election.cc:302] T 00000000000000000000000000000000 P f5029f037aa44a50994330e0f70b9bf2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f5029f037aa44a50994330e0f70b9bf2; no voters:
I20260811 19:03:40.889535 2836 leader_election.cc:288] T 00000000000000000000000000000000 P f5029f037aa44a50994330e0f70b9bf2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:03:40.889655 2849 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f5029f037aa44a50994330e0f70b9bf2 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:03:40.890337 30146 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 19:03:40.891168 2849 raft_consensus.cc:686] T 00000000000000000000000000000000 P f5029f037aa44a50994330e0f70b9bf2 [term 1 LEADER]: Becoming Leader. State: Replica: f5029f037aa44a50994330e0f70b9bf2, State: Running, Role: LEADER
I20260811 19:03:40.891803 2836 sys_catalog.cc:564] T 00000000000000000000000000000000 P f5029f037aa44a50994330e0f70b9bf2 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:03:40.891727 2849 consensus_queue.cc:237] T 00000000000000000000000000000000 P f5029f037aa44a50994330e0f70b9bf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5029f037aa44a50994330e0f70b9bf2" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 39809 } }
I20260811 19:03:40.895068 2852 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5029f037aa44a50994330e0f70b9bf2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f5029f037aa44a50994330e0f70b9bf2. Latest consensus state: current_term: 1 leader_uuid: "f5029f037aa44a50994330e0f70b9bf2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5029f037aa44a50994330e0f70b9bf2" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 39809 } } }
I20260811 19:03:40.895471 2852 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5029f037aa44a50994330e0f70b9bf2 [sys.catalog]: This master's current role is: LEADER
I20260811 19:03:40.895030 2851 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5029f037aa44a50994330e0f70b9bf2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f5029f037aa44a50994330e0f70b9bf2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5029f037aa44a50994330e0f70b9bf2" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 39809 } } }
I20260811 19:03:40.896055 2851 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5029f037aa44a50994330e0f70b9bf2 [sys.catalog]: This master's current role is: LEADER
I20260811 19:03:40.903342 2863 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:03:40.910331 2863 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:03:40.930675 2863 catalog_manager.cc:1347] Generated new cluster ID: c3a92cf5031c450b8a70b080272f1c04
I20260811 19:03:40.930903 2863 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:03:40.932144 2841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:40.907386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57475 (local address 127.29.112.190:39809)
0811 19:03:40.907604 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:40.907614 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:40.911073 (+ 3459us) server_negotiation.cc:407] Connection header received
0811 19:03:40.911406 (+ 333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:40.911412 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:40.911508 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:40.911589 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:40.913868 (+ 2279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:40.915133 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:40.916951 (+ 1818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:40.917601 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:40.917722 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:40.921866 (+ 4144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:03:40.921876 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:03:40.921883 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:03:40.921940 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:03:40.927014 (+ 5074us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:40.928896 (+ 1882us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:40.928903 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:40.928906 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:40.929143 (+ 237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:40.929776 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:40.929781 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:40.929784 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:40.930335 (+ 551us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:03:40.930691 (+ 356us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:40.931346 (+ 655us) server_negotiation.cc:299] Negotiation successful
0811 19:03:40.931639 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260811 19:03:40.952133 2863 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:03:40.953712 2863 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:03:40.966353 2863 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f5029f037aa44a50994330e0f70b9bf2: Generated new TSK 0
I20260811 19:03:40.967283 2863 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 19:03:40.980921 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 2710
I20260811 19:03:40.982220 2830 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:03:41.082857 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 2710
2026-08-11T19:03:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2564 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474615205622-30146-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:03:41 dist-test-slave-h6n0 krb5kdc[2888](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:03:41 dist-test-slave-h6n0 krb5kdc[2888](info): set up 2 sockets
Aug 11 19:03:41 dist-test-slave-h6n0 krb5kdc[2888](info): commencing operation
krb5kdc: starting...
W20260811 19:03:42.516896 30146 mini_kdc.cc:121] Time spent starting KDC: real 1.388s user 0.001s sys 0.049s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474615205622-30146-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474615205622-30146-0/krb5kdc/test-user.keytab.
Aug 11 19:03:42 dist-test-slave-h6n0 krb5kdc[2888](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475022, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:03:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:03:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.112.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab.
Entry for principal kudu/127.29.112.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab.
I20260811 19:03:42.895399 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474615205622-30146-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474615205622-30146-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474615205622-30146-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.190:44009
--webserver_interface=127.29.112.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab
--principal=kudu/127.29.112.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:39805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.190:44009
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:03:43.280002 2910 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:03:43.280287 2910 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:03:43.280390 2910 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:03:43.287683 2910 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:03:43.287811 2910 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:03:43.287863 2910 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:03:43.287921 2910 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:03:43.287977 2910 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:03:43.299038 2910 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:39805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474615205622-30146-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474615205622-30146-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.112.190:44009
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab
--principal=kudu/127.29.112.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.190:44009
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.112.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474615205622-30146-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:03:43.300625 2910 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:03:43.301985 2910 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:03:43.314620 2918 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:03:43.314641 2921 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:03:43.314590 2919 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:03:43.315634 2910 server_base.cc:1034] running on GCE node
I20260811 19:03:43.316524 2910 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:03:43.318085 2910 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:03:43.319407 2910 hybrid_clock.cc:648] HybridClock initialized: now 1786475023319364 us; error 58 us; skew 500 ppm
I20260811 19:03:43.319811 2910 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:03:43 dist-test-slave-h6n0 krb5kdc[2888](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475023, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:03:43.323422 2910 init.cc:377] Logged in from keytab as kudu/127.29.112.190@KRBTEST.COM (short username kudu)
I20260811 19:03:43.325080 2910 webserver.cc:460] Webserver started at http://127.29.112.190:46357/ using document root <none> and password file <none>
I20260811 19:03:43.325799 2910 fs_manager.cc:359] Metadata directory not provided
I20260811 19:03:43.325964 2910 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:03:43.326290 2910 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:03:43.330899 2910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474615205622-30146-0/minicluster-data/master-0/data/instance:
uuid: "3c2002b28fe945cd8868194d5648cd87"
format_stamp: "Formatted at 2026-08-11 19:03:43 on dist-test-slave-h6n0"
server_key: "668b3c813d5e6200c892012b78fad421"
server_key_iv: "a7c215805c88cc690c1c41dc16543692"
server_key_version: "encryptionkey@0"
I20260811 19:03:43.331976 2910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474615205622-30146-0/minicluster-data/master-0/wal/instance:
uuid: "3c2002b28fe945cd8868194d5648cd87"
format_stamp: "Formatted at 2026-08-11 19:03:43 on dist-test-slave-h6n0"
server_key: "668b3c813d5e6200c892012b78fad421"
server_key_iv: "a7c215805c88cc690c1c41dc16543692"
server_key_version: "encryptionkey@0"
I20260811 19:03:43.338991 2910 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.003s sys 0.004s
I20260811 19:03:43.343263 2931 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:03:43.344837 2910 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260811 19:03:43.345031 2910 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474615205622-30146-0/minicluster-data/master-0/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474615205622-30146-0/minicluster-data/master-0/wal
uuid: "3c2002b28fe945cd8868194d5648cd87"
format_stamp: "Formatted at 2026-08-11 19:03:43 on dist-test-slave-h6n0"
server_key: "668b3c813d5e6200c892012b78fad421"
server_key_iv: "a7c215805c88cc690c1c41dc16543692"
server_key_version: "encryptionkey@0"
I20260811 19:03:43.345253 2910 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474615205622-30146-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474615205622-30146-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474615205622-30146-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:03:43.363976 2910 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:03:43.368739 2910 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:03:43.369050 2910 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:03:43.390507 2910 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.190:44009
I20260811 19:03:43.390645 3022 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.190:44009 every 8 connection(s)
I20260811 19:03:43.392023 2910 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
I20260811 19:03:43.395782 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 2910
I20260811 19:03:43.396003 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474615205622-30146-0/minicluster-data/master-0/wal/instance
I20260811 19:03:43.396314 3024 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:03:43.396562 30146 external_mini_cluster.cc:1428] Setting key 4ca116ab1774482ae2b82b0152d0fe0b
I20260811 19:03:43.411878 3024 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c2002b28fe945cd8868194d5648cd87: Bootstrap starting.
I20260811 19:03:43.413499 3027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:43.398740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45527 (local address 127.29.112.190:44009)
0811 19:03:43.399613 (+ 873us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:43.399632 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:43.399682 (+ 50us) server_negotiation.cc:407] Connection header received
0811 19:03:43.401592 (+ 1910us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:43.401620 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:43.402148 (+ 528us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:43.402603 (+ 455us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:43.403487 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:43.405547 (+ 2060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:43.407569 (+ 2022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:43.408700 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:43.408912 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:43.410506 (+ 1594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:03:43.410517 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:03:43.410566 (+ 49us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:03:43.411310 (+ 744us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:03:43.411421 (+ 111us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:03:43.411461 (+ 40us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:03:43.411590 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:43.411899 (+ 309us) server_negotiation.cc:299] Negotiation successful
0811 19:03:43.412204 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":627,"thread_start_us":421,"threads_started":1}
I20260811 19:03:43.415603 3024 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3c2002b28fe945cd8868194d5648cd87: Neither blocks nor log segments found. Creating new log.
I20260811 19:03:43.416903 3024 log.cc:824] T 00000000000000000000000000000000 P 3c2002b28fe945cd8868194d5648cd87: Log is configured to *not* fsync() on all Append() calls
I20260811 19:03:43.417755 30146 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 19:03:43.421412 3024 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c2002b28fe945cd8868194d5648cd87: No bootstrap required, opened a new log
I20260811 19:03:43.430886 3027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:43.422831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38311 (local address 127.29.112.190:44009)
0811 19:03:43.423051 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:43.423058 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:43.423415 (+ 357us) server_negotiation.cc:407] Connection header received
0811 19:03:43.423581 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:43.423586 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:43.423667 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:43.423796 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:43.424745 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:43.425939 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:43.427556 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:43.428300 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:43.428461 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:43.429727 (+ 1266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:03:43.429734 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:03:43.429739 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:03:43.429769 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:03:43.429822 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:03:43.429843 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:03:43.429950 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:43.430318 (+ 368us) server_negotiation.cc:299] Negotiation successful
0811 19:03:43.430450 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260811 19:03:43.432216 3024 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3c2002b28fe945cd8868194d5648cd87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c2002b28fe945cd8868194d5648cd87" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 44009 } }
I20260811 19:03:43.432688 3024 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3c2002b28fe945cd8868194d5648cd87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:03:43.432837 3024 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3c2002b28fe945cd8868194d5648cd87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c2002b28fe945cd8868194d5648cd87, State: Initialized, Role: FOLLOWER
I20260811 19:03:43.433516 3024 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3c2002b28fe945cd8868194d5648cd87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c2002b28fe945cd8868194d5648cd87" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 44009 } }
I20260811 19:03:43.433806 3024 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3c2002b28fe945cd8868194d5648cd87 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:03:43.433992 3024 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3c2002b28fe945cd8868194d5648cd87 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:03:43.434191 3024 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3c2002b28fe945cd8868194d5648cd87 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:03:43.436599 3024 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3c2002b28fe945cd8868194d5648cd87 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c2002b28fe945cd8868194d5648cd87" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 44009 } }
I20260811 19:03:43.437155 3024 leader_election.cc:302] T 00000000000000000000000000000000 P 3c2002b28fe945cd8868194d5648cd87 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3c2002b28fe945cd8868194d5648cd87; no voters:
I20260811 19:03:43.438055 3024 leader_election.cc:288] T 00000000000000000000000000000000 P 3c2002b28fe945cd8868194d5648cd87 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:03:43.438133 3041 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3c2002b28fe945cd8868194d5648cd87 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:03:43.441304 3024 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3c2002b28fe945cd8868194d5648cd87 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:03:43.441907 3041 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3c2002b28fe945cd8868194d5648cd87 [term 1 LEADER]: Becoming Leader. State: Replica: 3c2002b28fe945cd8868194d5648cd87, State: Running, Role: LEADER
I20260811 19:03:43.442623 3041 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3c2002b28fe945cd8868194d5648cd87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c2002b28fe945cd8868194d5648cd87" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 44009 } }
I20260811 19:03:43.446434 3050 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c2002b28fe945cd8868194d5648cd87 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3c2002b28fe945cd8868194d5648cd87. Latest consensus state: current_term: 1 leader_uuid: "3c2002b28fe945cd8868194d5648cd87" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c2002b28fe945cd8868194d5648cd87" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 44009 } } }
I20260811 19:03:43.447088 3050 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c2002b28fe945cd8868194d5648cd87 [sys.catalog]: This master's current role is: LEADER
I20260811 19:03:43.447937 3047 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c2002b28fe945cd8868194d5648cd87 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3c2002b28fe945cd8868194d5648cd87" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c2002b28fe945cd8868194d5648cd87" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 44009 } } }
I20260811 19:03:43.448329 3065 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:03:43.448302 3047 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c2002b28fe945cd8868194d5648cd87 [sys.catalog]: This master's current role is: LEADER
I20260811 19:03:43.455708 3065 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:03:43.467391 3065 catalog_manager.cc:1347] Generated new cluster ID: 535b373e1ec74ba89e00fda9d5b84262
I20260811 19:03:43.467545 3065 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:03:43.489635 3065 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:03:43.491135 3065 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:03:43.503290 3065 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3c2002b28fe945cd8868194d5648cd87: Generated new TSK 0
I20260811 19:03:43.504899 3065 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 19:03:43.516232 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 2910
W20260811 19:03:43.517421 3014 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:45527
W20260811 19:03:43.518177 30146 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 2910: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 19:03:43.518347 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 2910
2026-08-11T19:03:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (2440 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474615205622-30146-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:03:43 dist-test-slave-h6n0 krb5kdc[3085](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:03:43 dist-test-slave-h6n0 krb5kdc[3085](info): set up 2 sockets
Aug 11 19:03:43 dist-test-slave-h6n0 krb5kdc[3085](info): commencing operation
krb5kdc: starting...
W20260811 19:03:44.969313 30146 mini_kdc.cc:121] Time spent starting KDC: real 1.399s user 0.004s sys 0.052s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474615205622-30146-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474615205622-30146-0/krb5kdc/test-user.keytab.
Aug 11 19:03:45 dist-test-slave-h6n0 krb5kdc[3085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475025, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:03:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:03:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.112.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab.
Entry for principal kudu/127.29.112.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab.
I20260811 19:03:45.306056 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474615205622-30146-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474615205622-30146-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474615205622-30146-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.190:38165
--webserver_interface=127.29.112.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab
--principal=kudu/127.29.112.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:42645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.190:38165
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:03:45.666122 3103 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:03:45.666464 3103 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:03:45.666654 3103 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:03:45.674301 3103 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:03:45.674441 3103 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:03:45.674535 3103 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:03:45.674654 3103 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:03:45.674697 3103 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:03:45.685284 3103 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:42645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474615205622-30146-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474615205622-30146-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.112.190:38165
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab
--principal=kudu/127.29.112.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.190:38165
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.112.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474615205622-30146-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:03:45.686703 3103 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:03:45.688091 3103 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:03:45.701540 3110 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:03:45.701831 3111 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:03:45.702337 3113 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:03:45.702925 3103 server_base.cc:1034] running on GCE node
I20260811 19:03:45.704005 3103 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:03:45.705555 3103 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:03:45.706809 3103 hybrid_clock.cc:648] HybridClock initialized: now 1786475025706810 us; error 114 us; skew 500 ppm
I20260811 19:03:45.707216 3103 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:03:45 dist-test-slave-h6n0 krb5kdc[3085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475025, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:03:45.711011 3103 init.cc:377] Logged in from keytab as kudu/127.29.112.190@KRBTEST.COM (short username kudu)
I20260811 19:03:45.712656 3103 webserver.cc:460] Webserver started at http://127.29.112.190:37945/ using document root <none> and password file <none>
I20260811 19:03:45.713400 3103 fs_manager.cc:359] Metadata directory not provided
I20260811 19:03:45.713564 3103 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:03:45.713900 3103 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:03:45.717113 3103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474615205622-30146-0/minicluster-data/master-0/data/instance:
uuid: "9045f0f3f470436f9b6bed0abcd9cb47"
format_stamp: "Formatted at 2026-08-11 19:03:45 on dist-test-slave-h6n0"
server_key: "27a93986f044c1729e5993c55fbbbb71"
server_key_iv: "afc8006af0d70376f418dca33e93a2b1"
server_key_version: "encryptionkey@0"
I20260811 19:03:45.717924 3103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474615205622-30146-0/minicluster-data/master-0/wal/instance:
uuid: "9045f0f3f470436f9b6bed0abcd9cb47"
format_stamp: "Formatted at 2026-08-11 19:03:45 on dist-test-slave-h6n0"
server_key: "27a93986f044c1729e5993c55fbbbb71"
server_key_iv: "afc8006af0d70376f418dca33e93a2b1"
server_key_version: "encryptionkey@0"
I20260811 19:03:45.724900 3103 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.004s
I20260811 19:03:45.728689 3128 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:03:45.730060 3103 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 19:03:45.730266 3103 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474615205622-30146-0/minicluster-data/master-0/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474615205622-30146-0/minicluster-data/master-0/wal
uuid: "9045f0f3f470436f9b6bed0abcd9cb47"
format_stamp: "Formatted at 2026-08-11 19:03:45 on dist-test-slave-h6n0"
server_key: "27a93986f044c1729e5993c55fbbbb71"
server_key_iv: "afc8006af0d70376f418dca33e93a2b1"
server_key_version: "encryptionkey@0"
I20260811 19:03:45.730494 3103 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474615205622-30146-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474615205622-30146-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474615205622-30146-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:03:45.736591 3103 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:03:45.741257 3103 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:03:45.741600 3103 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:03:45.763904 3103 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.190:38165
I20260811 19:03:45.763955 3211 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.190:38165 every 8 connection(s)
I20260811 19:03:45.765378 3103 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
I20260811 19:03:45.769539 3213 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:03:45.772083 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 3103
I20260811 19:03:45.772289 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474615205622-30146-0/minicluster-data/master-0/wal/instance
I20260811 19:03:45.772830 30146 external_mini_cluster.cc:1428] Setting key 0d8313acda6eeb58b473b9ef7591915b
I20260811 19:03:45.782174 3217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:45.776032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36577 (local address 127.29.112.190:38165)
0811 19:03:45.776826 (+ 794us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:45.776842 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:45.776886 (+ 44us) server_negotiation.cc:407] Connection header received
0811 19:03:45.778692 (+ 1806us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:45.778715 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:45.779052 (+ 337us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:45.779437 (+ 385us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:45.780513 (+ 1076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:03:45.780519 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:03:45.780534 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:03:45.780550 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:03:45.780612 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:03:45.780631 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:03:45.780919 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:45.781084 (+ 165us) server_negotiation.cc:299] Negotiation successful
0811 19:03:45.781257 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":580,"thread_start_us":357,"threads_started":1}
I20260811 19:03:45.783583 3213 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9045f0f3f470436f9b6bed0abcd9cb47: Bootstrap starting.
I20260811 19:03:45.786936 3213 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9045f0f3f470436f9b6bed0abcd9cb47: Neither blocks nor log segments found. Creating new log.
I20260811 19:03:45.786903 30146 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 19:03:45.787974 3213 log.cc:824] T 00000000000000000000000000000000 P 9045f0f3f470436f9b6bed0abcd9cb47: Log is configured to *not* fsync() on all Append() calls
I20260811 19:03:45.791725 3213 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9045f0f3f470436f9b6bed0abcd9cb47: No bootstrap required, opened a new log
I20260811 19:03:45.795578 3217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:45.792098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50907 (local address 127.29.112.190:38165)
0811 19:03:45.792338 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:45.792349 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:45.792930 (+ 581us) server_negotiation.cc:407] Connection header received
0811 19:03:45.793099 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:45.793105 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:45.793183 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:45.793351 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:45.794516 (+ 1165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:03:45.794523 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:03:45.794638 (+ 115us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:03:45.794680 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:03:45.794748 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:03:45.794783 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:03:45.794993 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:45.795139 (+ 146us) server_negotiation.cc:299] Negotiation successful
0811 19:03:45.795209 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 19:03:45.801702 3213 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9045f0f3f470436f9b6bed0abcd9cb47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9045f0f3f470436f9b6bed0abcd9cb47" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 38165 } }
I20260811 19:03:45.802084 3213 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9045f0f3f470436f9b6bed0abcd9cb47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:03:45.802167 3213 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9045f0f3f470436f9b6bed0abcd9cb47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9045f0f3f470436f9b6bed0abcd9cb47, State: Initialized, Role: FOLLOWER
I20260811 19:03:45.802812 3213 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9045f0f3f470436f9b6bed0abcd9cb47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9045f0f3f470436f9b6bed0abcd9cb47" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 38165 } }
I20260811 19:03:45.803102 3213 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9045f0f3f470436f9b6bed0abcd9cb47 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:03:45.803293 3213 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9045f0f3f470436f9b6bed0abcd9cb47 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:03:45.803514 3213 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9045f0f3f470436f9b6bed0abcd9cb47 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:03:45.805894 3213 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9045f0f3f470436f9b6bed0abcd9cb47 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9045f0f3f470436f9b6bed0abcd9cb47" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 38165 } }
I20260811 19:03:45.806474 3213 leader_election.cc:302] T 00000000000000000000000000000000 P 9045f0f3f470436f9b6bed0abcd9cb47 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9045f0f3f470436f9b6bed0abcd9cb47; no voters:
I20260811 19:03:45.807351 3213 leader_election.cc:288] T 00000000000000000000000000000000 P 9045f0f3f470436f9b6bed0abcd9cb47 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:03:45.807835 3228 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9045f0f3f470436f9b6bed0abcd9cb47 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:03:45.808970 3228 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9045f0f3f470436f9b6bed0abcd9cb47 [term 1 LEADER]: Becoming Leader. State: Replica: 9045f0f3f470436f9b6bed0abcd9cb47, State: Running, Role: LEADER
I20260811 19:03:45.809526 3228 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9045f0f3f470436f9b6bed0abcd9cb47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9045f0f3f470436f9b6bed0abcd9cb47" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 38165 } }
I20260811 19:03:45.809892 3213 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9045f0f3f470436f9b6bed0abcd9cb47 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:03:45.812481 3230 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9045f0f3f470436f9b6bed0abcd9cb47 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9045f0f3f470436f9b6bed0abcd9cb47. Latest consensus state: current_term: 1 leader_uuid: "9045f0f3f470436f9b6bed0abcd9cb47" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9045f0f3f470436f9b6bed0abcd9cb47" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 38165 } } }
I20260811 19:03:45.812980 3230 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9045f0f3f470436f9b6bed0abcd9cb47 [sys.catalog]: This master's current role is: LEADER
I20260811 19:03:45.813560 3229 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9045f0f3f470436f9b6bed0abcd9cb47 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9045f0f3f470436f9b6bed0abcd9cb47" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9045f0f3f470436f9b6bed0abcd9cb47" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 38165 } } }
I20260811 19:03:45.813897 3229 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9045f0f3f470436f9b6bed0abcd9cb47 [sys.catalog]: This master's current role is: LEADER
I20260811 19:03:45.814009 3246 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:03:45.819597 3246 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:03:45.829659 3246 catalog_manager.cc:1347] Generated new cluster ID: 9bfe2e2ef729448db3119d2591a04592
I20260811 19:03:45.829934 3246 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:03:45.844981 3246 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:03:45.845461 3246 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:03:45.860138 3246 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9045f0f3f470436f9b6bed0abcd9cb47: Generated new TSK 0
I20260811 19:03:45.861006 3246 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 19:03:45.867578 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 3103
W20260811 19:03:45.869087 3203 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:36577
W20260811 19:03:45.869796 30146 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 3103: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 19:03:45.869946 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 3103
2026-08-11T19:03:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (2350 ms)
[----------] 18 tests from AuthTokenIssuingTest (45067 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474615205622-30146-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:03:46 dist-test-slave-h6n0 krb5kdc[3261](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:03:46 dist-test-slave-h6n0 krb5kdc[3261](info): set up 2 sockets
Aug 11 19:03:46 dist-test-slave-h6n0 krb5kdc[3261](info): commencing operation
krb5kdc: starting...
W20260811 19:03:47.306648 30146 mini_kdc.cc:121] Time spent starting KDC: real 1.386s user 0.004s sys 0.053s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474615205622-30146-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474615205622-30146-0/krb5kdc/test-user.keytab.
Aug 11 19:03:47 dist-test-slave-h6n0 krb5kdc[3261](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475027, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:03:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:03:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.112.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab.
Entry for principal kudu/127.29.112.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab.
I20260811 19:03:47.647133 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474615205622-30146-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474615205622-30146-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474615205622-30146-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.190:36653
--webserver_interface=127.29.112.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab
--principal=kudu/127.29.112.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:40041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.190:36653,127.29.112.189:35303,127.29.112.188:35753
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:03:48.001538 3287 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 19:03:48.001956 3287 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:03:48.002094 3287 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:03:48.002238 3287 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:03:48.009800 3287 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:03:48.009941 3287 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:03:48.010061 3287 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:03:48.010156 3287 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:03:48.010252 3287 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:03:48.020748 3287 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:40041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474615205622-30146-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474615205622-30146-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.112.190:36653,127.29.112.189:35303,127.29.112.188:35753
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab
--principal=kudu/127.29.112.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.190:36653
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.112.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474615205622-30146-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:03:48.022113 3287 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:03:48.023521 3287 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:03:48.037042 3298 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:03:48.037099 3294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:03:48.037104 3293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:03:48.038306 3287 server_base.cc:1034] running on GCE node
I20260811 19:03:48.039201 3287 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:03:48.040733 3287 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:03:48.042021 3287 hybrid_clock.cc:648] HybridClock initialized: now 1786475028041958 us; error 41 us; skew 500 ppm
I20260811 19:03:48.042376 3287 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:03:48 dist-test-slave-h6n0 krb5kdc[3261](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475028, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:03:48.051311 3287 init.cc:377] Logged in from keytab as kudu/127.29.112.190@KRBTEST.COM (short username kudu)
I20260811 19:03:48.053323 3287 webserver.cc:460] Webserver started at http://127.29.112.190:41735/ using document root <none> and password file <none>
I20260811 19:03:48.054139 3287 fs_manager.cc:359] Metadata directory not provided
I20260811 19:03:48.054281 3287 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:03:48.054709 3287 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:03:48.057785 3287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474615205622-30146-0/minicluster-data/master-0/data/instance:
uuid: "1217462373bb43d08e40cfbba99b8378"
format_stamp: "Formatted at 2026-08-11 19:03:48 on dist-test-slave-h6n0"
server_key: "26031e50427a86bbbe7110b85485a9be"
server_key_iv: "d36f027c1e967120583a0e09d092a63d"
server_key_version: "encryptionkey@0"
I20260811 19:03:48.058688 3287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474615205622-30146-0/minicluster-data/master-0/wal/instance:
uuid: "1217462373bb43d08e40cfbba99b8378"
format_stamp: "Formatted at 2026-08-11 19:03:48 on dist-test-slave-h6n0"
server_key: "26031e50427a86bbbe7110b85485a9be"
server_key_iv: "d36f027c1e967120583a0e09d092a63d"
server_key_version: "encryptionkey@0"
I20260811 19:03:48.063879 3287 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260811 19:03:48.067376 3310 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:03:48.069298 3287 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 19:03:48.069551 3287 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474615205622-30146-0/minicluster-data/master-0/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474615205622-30146-0/minicluster-data/master-0/wal
uuid: "1217462373bb43d08e40cfbba99b8378"
format_stamp: "Formatted at 2026-08-11 19:03:48 on dist-test-slave-h6n0"
server_key: "26031e50427a86bbbe7110b85485a9be"
server_key_iv: "d36f027c1e967120583a0e09d092a63d"
server_key_version: "encryptionkey@0"
I20260811 19:03:48.069823 3287 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474615205622-30146-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474615205622-30146-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474615205622-30146-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:03:48.085178 3287 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:03:48.089687 3287 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:03:48.089922 3287 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:03:48.110412 3287 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.190:36653
I20260811 19:03:48.110432 3394 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.190:36653 every 8 connection(s)
I20260811 19:03:48.111871 3287 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
I20260811 19:03:48.115634 3398 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:03:48.119050 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 3287
I20260811 19:03:48.119266 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474615205622-30146-0/minicluster-data/master-0/wal/instance
I20260811 19:03:48.120098 30146 external_mini_cluster.cc:1428] Setting key 0c29347a6850ac91945b3a927eaf8394
I20260811 19:03:48.124328 3398 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.112.190" port: 36653 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 19:03:48 dist-test-slave-h6n0 krb5kdc[3261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475028, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.190@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 19:03:48.154506 3400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:48.128256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:36653 (local address 127.0.0.1:36956)
0811 19:03:48.129548 (+ 1292us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:48.129602 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 19:03:48.131377 (+ 1775us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:03:48.132306 (+ 929us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:03:48.132311 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:03:48.132886 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:03:48.133221 (+ 335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:03:48.133241 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:03:48.135436 (+ 2195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:03:48.135445 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:03:48.137336 (+ 1891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:03:48.137354 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:03:48.138445 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:03:48.138454 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:03:48.138706 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:48.139777 (+ 1071us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:03:48.139811 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:03:48.142795 (+ 2984us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:03:48.149141 (+ 6346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:03:48.149152 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:03:48.149162 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:03:48.149624 (+ 462us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:03:48.150143 (+ 519us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:03:48.150147 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:03:48.150149 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:03:48.150323 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:03:48.151518 (+ 1195us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:03:48.151528 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:03:48.151914 (+ 386us) client_negotiation.cc:769] Sending connection context
0811 19:03:48.152338 (+ 424us) client_negotiation.cc:240] Negotiation successful
0811 19:03:48.153243 (+ 905us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1038,"mutex_wait_us":209,"thread_start_us":825,"threads_started":1}
I20260811 19:03:48.154680 3402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:48.128256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36956 (local address 127.29.112.190:36653)
0811 19:03:48.129156 (+ 900us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:48.129174 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:48.129741 (+ 567us) server_negotiation.cc:407] Connection header received
0811 19:03:48.131740 (+ 1999us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:48.131751 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:48.132056 (+ 305us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:48.132268 (+ 212us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:48.133438 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:48.135246 (+ 1808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:48.137542 (+ 2296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:48.138141 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:48.138383 (+ 242us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:48.143254 (+ 4871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:03:48.143273 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:03:48.143301 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:03:48.143372 (+ 71us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:03:48.148966 (+ 5594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:48.149771 (+ 805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:48.149783 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:48.149790 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:48.150016 (+ 226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:48.150438 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:48.150446 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:48.150452 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:48.151251 (+ 799us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:03:48.151513 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:48.152437 (+ 924us) server_negotiation.cc:299] Negotiation successful
0811 19:03:48.152833 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":629,"thread_start_us":387,"threads_started":1}
I20260811 19:03:48.157990 3398 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.112.189" port: 35303 } has no permanent_uuid. Determining permanent_uuid...
I20260811 19:03:48.159409 3400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:48.158819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.189:35303 (local address 127.0.0.1:37398)
0811 19:03:48.159040 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:48.159219 (+ 179us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.189:35303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260811 19:03:48.159730 3312 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.112.189:35303: connect: Connection refused (error 111)
I20260811 19:03:48.161481 3400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:48.161006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.189:35303 (local address 127.0.0.1:37404)
0811 19:03:48.161206 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:48.161309 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.189:35303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
WARNING: no policy specified for kudu/127.29.112.189@KRBTEST.COM; defaulting to no policy
W20260811 19:03:48.161713 3398 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.112.189:35303: Network error: Client connection negotiation failed: client connection to 127.29.112.189:35303: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.189.keytab.
Entry for principal kudu/127.29.112.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.189.keytab.
I20260811 19:03:48.210435 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474615205622-30146-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474615205622-30146-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474615205622-30146-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474615205622-30146-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.189:35303
--webserver_interface=127.29.112.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.189.keytab
--principal=kudu/127.29.112.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:40041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.190:36653,127.29.112.189:35303,127.29.112.188:35753
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 19:03:48.211076 3398 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.112.189" port: 35303 } attempt: 1
I20260811 19:03:48.212358 3400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:48.211873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.189:35303 (local address 127.0.0.1:37420)
0811 19:03:48.212100 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:48.212222 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.189:35303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260811 19:03:48.213912 3400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:48.213312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.189:35303 (local address 127.0.0.1:37424)
0811 19:03:48.213570 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:48.213689 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.189:35303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
W20260811 19:03:48.214206 3398 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.112.189:35303: Network error: Client connection negotiation failed: client connection to 127.29.112.189:35303: connect: Connection refused (error 111)
I20260811 19:03:48.282732 3398 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.112.189" port: 35303 } attempt: 2
I20260811 19:03:48.284606 3400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:48.283744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.189:35303 (local address 127.0.0.1:37428)
0811 19:03:48.284168 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:48.284341 (+ 173us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.189:35303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115}
I20260811 19:03:48.286018 3400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:48.285558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.189:35303 (local address 127.0.0.1:37440)
0811 19:03:48.285753 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:48.285841 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.189:35303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260811 19:03:48.286269 3398 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.112.189:35303: Network error: Client connection negotiation failed: client connection to 127.29.112.189:35303: connect: Connection refused (error 111)
I20260811 19:03:48.377636 3398 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.112.189" port: 35303 } attempt: 3
I20260811 19:03:48.379350 3400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:48.378770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.189:35303 (local address 127.0.0.1:37454)
0811 19:03:48.379042 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:48.379160 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.189:35303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
I20260811 19:03:48.380723 3400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:48.380305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.189:35303 (local address 127.0.0.1:37468)
0811 19:03:48.380488 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:48.380573 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.189:35303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
W20260811 19:03:48.380991 3398 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.112.189:35303: Network error: Client connection negotiation failed: client connection to 127.29.112.189:35303: connect: Connection refused (error 111)
I20260811 19:03:48.524369 3398 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.112.189" port: 35303 } attempt: 4
I20260811 19:03:48.525770 3400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:48.525188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.189:35303 (local address 127.0.0.1:37474)
0811 19:03:48.525520 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:48.525624 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.189:35303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193}
I20260811 19:03:48.526978 3400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:48.526613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.189:35303 (local address 127.0.0.1:37488)
0811 19:03:48.526760 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:48.526853 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.189:35303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260811 19:03:48.527197 3398 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.112.189:35303: Network error: Client connection negotiation failed: client connection to 127.29.112.189:35303: connect: Connection refused (error 111)
W20260811 19:03:48.598029 3408 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 19:03:48.598810 3408 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:03:48.598912 3408 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:03:48.599046 3408 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:03:48.606966 3408 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:03:48.607057 3408 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:03:48.607143 3408 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:03:48.607232 3408 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:03:48.607342 3408 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:03:48.619724 3408 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:40041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474615205622-30146-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474615205622-30146-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.29.112.190:36653,127.29.112.189:35303,127.29.112.188:35753
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.189.keytab
--principal=kudu/127.29.112.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.189:35303
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474615205622-30146-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.29.112.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474615205622-30146-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:03:48.621157 3408 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:03:48.622563 3408 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:03:48.634548 3418 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:03:48.634616 3417 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:03:48.634639 3420 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:03:48.636108 3408 server_base.cc:1034] running on GCE node
I20260811 19:03:48.636849 3408 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:03:48.638703 3408 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:03:48.639956 3408 hybrid_clock.cc:648] HybridClock initialized: now 1786475028639889 us; error 44 us; skew 500 ppm
I20260811 19:03:48.640285 3408 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:03:48 dist-test-slave-h6n0 krb5kdc[3261](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475028, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:03:48.643757 3408 init.cc:377] Logged in from keytab as kudu/127.29.112.189@KRBTEST.COM (short username kudu)
I20260811 19:03:48.645679 3408 webserver.cc:460] Webserver started at http://127.29.112.189:40905/ using document root <none> and password file <none>
I20260811 19:03:48.646736 3408 fs_manager.cc:359] Metadata directory not provided
I20260811 19:03:48.646860 3408 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:03:48.647190 3408 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:03:48.650213 3408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474615205622-30146-0/minicluster-data/master-1/data/instance:
uuid: "66b539a9a527407181be314b47238b2f"
format_stamp: "Formatted at 2026-08-11 19:03:48 on dist-test-slave-h6n0"
server_key: "0cc18e9108e30c188da71876af9729a5"
server_key_iv: "a5934ab016aa1427b19c2bb7f1c8f747"
server_key_version: "encryptionkey@0"
I20260811 19:03:48.651057 3408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474615205622-30146-0/minicluster-data/master-1/wal/instance:
uuid: "66b539a9a527407181be314b47238b2f"
format_stamp: "Formatted at 2026-08-11 19:03:48 on dist-test-slave-h6n0"
server_key: "0cc18e9108e30c188da71876af9729a5"
server_key_iv: "a5934ab016aa1427b19c2bb7f1c8f747"
server_key_version: "encryptionkey@0"
I20260811 19:03:48.656085 3408 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 19:03:48.659927 3432 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:03:48.661186 3408 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 19:03:48.661370 3408 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474615205622-30146-0/minicluster-data/master-1/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474615205622-30146-0/minicluster-data/master-1/wal
uuid: "66b539a9a527407181be314b47238b2f"
format_stamp: "Formatted at 2026-08-11 19:03:48 on dist-test-slave-h6n0"
server_key: "0cc18e9108e30c188da71876af9729a5"
server_key_iv: "a5934ab016aa1427b19c2bb7f1c8f747"
server_key_version: "encryptionkey@0"
I20260811 19:03:48.661599 3408 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474615205622-30146-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474615205622-30146-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474615205622-30146-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:03:48.686463 3408 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:03:48.691174 3408 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:03:48.691471 3408 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:03:48.713627 3408 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.189:35303
I20260811 19:03:48.713666 3505 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.189:35303 every 8 connection(s)
I20260811 19:03:48.715134 3408 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474615205622-30146-0/minicluster-data/master-1/data/info.pb
I20260811 19:03:48.718039 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 3408
I20260811 19:03:48.718226 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474615205622-30146-0/minicluster-data/master-1/wal/instance
I20260811 19:03:48.718829 30146 external_mini_cluster.cc:1428] Setting key 26eba4bb22c92632a78d325c85bd038f
I20260811 19:03:48.719314 3507 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:03:48.729426 3507 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.112.190" port: 36653 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 19:03:48 dist-test-slave-h6n0 krb5kdc[3261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475028, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.189@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 19:03:48.754242 3509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:48.732884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36968 (local address 127.29.112.190:36653)
0811 19:03:48.733441 (+ 557us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:48.733450 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:48.734418 (+ 968us) server_negotiation.cc:407] Connection header received
0811 19:03:48.737155 (+ 2737us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:48.737160 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:48.737226 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:48.737331 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:48.739541 (+ 2210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:48.740277 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:48.742814 (+ 2537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:48.743223 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:48.743348 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:48.747931 (+ 4583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:03:48.747937 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:03:48.747941 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:03:48.747993 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:03:48.751037 (+ 3044us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:48.751929 (+ 892us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:48.751939 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:48.751942 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:48.752061 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:48.752494 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:48.752498 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:48.752501 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:48.752786 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:03:48.752948 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:48.753638 (+ 690us) server_negotiation.cc:299] Negotiation successful
0811 19:03:48.753858 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":273,"threads_started":1}
WARNING: no policy specified for kudu/127.29.112.188@KRBTEST.COM; defaulting to no policy
I20260811 19:03:48.755285 3511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:48.733432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:36653 (local address 127.0.0.1:36968)
0811 19:03:48.734235 (+ 803us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:48.734281 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 19:03:48.736852 (+ 2571us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:03:48.737591 (+ 739us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:03:48.737609 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:03:48.738471 (+ 862us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:03:48.739281 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:03:48.739308 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:03:48.740442 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:03:48.740450 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:03:48.742643 (+ 2193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:03:48.742661 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:03:48.743466 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:03:48.743476 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:03:48.743800 (+ 324us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:48.744626 (+ 826us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:03:48.744661 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:03:48.747644 (+ 2983us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:03:48.751198 (+ 3554us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:03:48.751208 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:03:48.751287 (+ 79us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:03:48.751769 (+ 482us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:03:48.752175 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:03:48.752179 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:03:48.752183 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:03:48.752373 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:03:48.752922 (+ 549us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:03:48.752931 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:03:48.753270 (+ 339us) client_negotiation.cc:769] Sending connection context
0811 19:03:48.753532 (+ 262us) client_negotiation.cc:240] Negotiation successful
0811 19:03:48.754066 (+ 534us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":348,"threads_started":1}
I20260811 19:03:48.757835 3507 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.112.189" port: 35303 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.188@KRBTEST.COM" created.
Aug 11 19:03:48 dist-test-slave-h6n0 krb5kdc[3261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475028, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.189@KRBTEST.COM for kudu/127.29.112.189@KRBTEST.COM
I20260811 19:03:48.777006 3514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:48.759093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37502 (local address 127.29.112.189:35303)
0811 19:03:48.759815 (+ 722us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:48.759821 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:48.759846 (+ 25us) server_negotiation.cc:407] Connection header received
0811 19:03:48.759888 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:48.759893 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:48.760071 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:48.760187 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:48.761233 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:48.762306 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:48.764646 (+ 2340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:48.765239 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:48.765408 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:48.769416 (+ 4008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:03:48.769433 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:03:48.769453 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:03:48.769495 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:03:48.773610 (+ 4115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:48.774433 (+ 823us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:48.774445 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:48.774450 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:48.774644 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:48.775068 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:48.775074 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:48.775079 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:48.775595 (+ 516us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:03:48.775796 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:48.776231 (+ 435us) server_negotiation.cc:299] Negotiation successful
0811 19:03:48.776691 (+ 460us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":127,"server-negotiator.queue_time_us":573,"thread_start_us":421,"threads_started":1}
I20260811 19:03:48.777100 3511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:48.758754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.189:35303 (local address 127.0.0.1:37502)
0811 19:03:48.758960 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:48.758987 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 19:03:48.759110 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:03:48.760248 (+ 1138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:03:48.760255 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:03:48.760798 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:03:48.761025 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:03:48.761039 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:03:48.762466 (+ 1427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:03:48.762474 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:03:48.764387 (+ 1913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:03:48.764425 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:03:48.765449 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:03:48.765458 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:03:48.765709 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:48.766381 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:03:48.766408 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:03:48.769181 (+ 2773us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:03:48.773766 (+ 4585us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:03:48.773770 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:03:48.773773 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:03:48.774291 (+ 518us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:03:48.774789 (+ 498us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:03:48.774794 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:03:48.774796 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:03:48.774955 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:03:48.775747 (+ 792us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:03:48.775755 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:03:48.775947 (+ 192us) client_negotiation.cc:769] Sending connection context
0811 19:03:48.776107 (+ 160us) client_negotiation.cc:240] Negotiation successful
0811 19:03:48.776448 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65}
I20260811 19:03:48.780236 3507 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.112.188" port: 35753 } has no permanent_uuid. Determining permanent_uuid...
I20260811 19:03:48.782415 3511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:48.781404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.188:35753 (local address 127.0.0.1:51142)
0811 19:03:48.781946 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:48.782175 (+ 229us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.188:35753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190}
W20260811 19:03:48.782961 3435 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.112.188:35753: connect: Connection refused (error 111)
I20260811 19:03:48.785478 3511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:48.784948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.188:35753 (local address 127.0.0.1:51146)
0811 19:03:48.785207 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:48.785326 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.188:35753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
W20260811 19:03:48.785856 3507 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.112.188:35753: Network error: Client connection negotiation failed: client connection to 127.29.112.188:35753: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.188.keytab.
Entry for principal kudu/127.29.112.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.188.keytab.
I20260811 19:03:48.805681 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474615205622-30146-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474615205622-30146-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474615205622-30146-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474615205622-30146-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.188:35753
--webserver_interface=127.29.112.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.188.keytab
--principal=kudu/127.29.112.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:40041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.190:36653,127.29.112.189:35303,127.29.112.188:35753
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 19:03:48.826758 3398 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.112.189" port: 35303 } attempt: 5
I20260811 19:03:48.835371 3507 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.112.188" port: 35753 } attempt: 1
Aug 11 19:03:48 dist-test-slave-h6n0 krb5kdc[3261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475028, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.190@KRBTEST.COM for kudu/127.29.112.189@KRBTEST.COM
I20260811 19:03:48.836465 3511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:48.835970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.188:35753 (local address 127.0.0.1:51152)
0811 19:03:48.836141 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:48.836254 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.188:35753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260811 19:03:48.837797 3511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:48.837276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.188:35753 (local address 127.0.0.1:51162)
0811 19:03:48.837503 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:48.837629 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.188:35753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260811 19:03:48.838001 3507 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.112.188:35753: Network error: Client connection negotiation failed: client connection to 127.29.112.188:35753: connect: Connection refused (error 111)
I20260811 19:03:48.843994 3514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:48.828002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37516 (local address 127.29.112.189:35303)
0811 19:03:48.828254 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:48.828262 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:48.828288 (+ 26us) server_negotiation.cc:407] Connection header received
0811 19:03:48.828545 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:48.828554 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:48.828653 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:48.828775 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:48.829675 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:48.830879 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:48.832093 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:48.832878 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:48.833018 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:48.837008 (+ 3990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:03:48.837012 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:03:48.837016 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:03:48.837046 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:03:48.840149 (+ 3103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:48.841547 (+ 1398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:48.841555 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:48.841559 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:48.841740 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:48.842257 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:48.842337 (+ 80us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:48.842341 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:48.842769 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:03:48.842927 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:48.843591 (+ 664us) server_negotiation.cc:299] Negotiation successful
0811 19:03:48.843770 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260811 19:03:48.844424 3400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:48.827818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.189:35303 (local address 127.0.0.1:37516)
0811 19:03:48.828082 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:48.828108 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 19:03:48.828263 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:03:48.828824 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:03:48.828829 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:03:48.829327 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:03:48.829521 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:03:48.829531 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:03:48.831034 (+ 1503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:03:48.831039 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:03:48.831930 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:03:48.831942 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:03:48.833054 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:03:48.833059 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:03:48.833231 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:48.833921 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:03:48.833939 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:03:48.836808 (+ 2869us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:03:48.840528 (+ 3720us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:03:48.840536 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:03:48.840542 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:03:48.841365 (+ 823us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:03:48.841880 (+ 515us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:03:48.841885 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:03:48.841889 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:03:48.842057 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:03:48.842969 (+ 912us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:03:48.842978 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:03:48.843215 (+ 237us) client_negotiation.cc:769] Sending connection context
0811 19:03:48.843460 (+ 245us) client_negotiation.cc:240] Negotiation successful
0811 19:03:48.843875 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90}
I20260811 19:03:48.845690 3398 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.112.188" port: 35753 } has no permanent_uuid. Determining permanent_uuid...
I20260811 19:03:48.846829 3400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:48.846405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.188:35753 (local address 127.0.0.1:51172)
0811 19:03:48.846614 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:48.846694 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.188:35753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260811 19:03:48.847723 3400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:48.847410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.188:35753 (local address 127.0.0.1:51184)
0811 19:03:48.847539 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:48.847628 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.188:35753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 19:03:48.847978 3398 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.112.188:35753: Network error: Client connection negotiation failed: client connection to 127.29.112.188:35753: connect: Connection refused (error 111)
I20260811 19:03:48.899339 3398 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.112.188" port: 35753 } attempt: 1
I20260811 19:03:48.901276 3400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:48.900545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.188:35753 (local address 127.0.0.1:51198)
0811 19:03:48.900771 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:48.900891 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.188:35753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260811 19:03:48.902819 3400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:48.902301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.188:35753 (local address 127.0.0.1:51212)
0811 19:03:48.902551 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:48.902705 (+ 154us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.188:35753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
W20260811 19:03:48.903074 3398 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.112.188:35753: Network error: Client connection negotiation failed: client connection to 127.29.112.188:35753: connect: Connection refused (error 111)
I20260811 19:03:48.906251 3507 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.112.188" port: 35753 } attempt: 2
I20260811 19:03:48.907449 3511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:48.906997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.188:35753 (local address 127.0.0.1:51222)
0811 19:03:48.907219 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:48.907301 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.188:35753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
I20260811 19:03:48.908573 3511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:48.908228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.188:35753 (local address 127.0.0.1:51234)
0811 19:03:48.908389 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:48.908465 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.188:35753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260811 19:03:48.908878 3507 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.112.188:35753: Network error: Client connection negotiation failed: client connection to 127.29.112.188:35753: connect: Connection refused (error 111)
I20260811 19:03:48.971354 3398 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.112.188" port: 35753 } attempt: 2
I20260811 19:03:48.972697 3400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:48.972177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.188:35753 (local address 127.0.0.1:33008)
0811 19:03:48.972442 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:48.972530 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.188:35753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260811 19:03:48.973769 3400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:48.973430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.188:35753 (local address 127.0.0.1:33020)
0811 19:03:48.973589 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:48.973668 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.188:35753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260811 19:03:48.974026 3398 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.112.188:35753: Network error: Client connection negotiation failed: client connection to 127.29.112.188:35753: connect: Connection refused (error 111)
I20260811 19:03:49.000138 3507 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.112.188" port: 35753 } attempt: 3
I20260811 19:03:49.001529 3511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:49.001061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.188:35753 (local address 127.0.0.1:33024)
0811 19:03:49.001269 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:49.001360 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.188:35753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260811 19:03:49.002920 3511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:49.002419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.188:35753 (local address 127.0.0.1:33032)
0811 19:03:49.002723 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:49.002815 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.188:35753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218}
W20260811 19:03:49.003495 3507 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.112.188:35753: Network error: Client connection negotiation failed: client connection to 127.29.112.188:35753: connect: Connection refused (error 111)
I20260811 19:03:49.080305 3398 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.112.188" port: 35753 } attempt: 3
I20260811 19:03:49.081861 3400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:49.081117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.188:35753 (local address 127.0.0.1:33046)
0811 19:03:49.081477 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:49.081626 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.188:35753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155}
I20260811 19:03:49.083602 3400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:49.083018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.188:35753 (local address 127.0.0.1:33052)
0811 19:03:49.083260 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:49.083448 (+ 188us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.188:35753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118}
W20260811 19:03:49.083974 3398 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.112.188:35753: Network error: Client connection negotiation failed: client connection to 127.29.112.188:35753: connect: Connection refused (error 111)
I20260811 19:03:49.146863 3507 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.112.188" port: 35753 } attempt: 4
I20260811 19:03:49.148473 3511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:49.147918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.188:35753 (local address 127.0.0.1:33060)
0811 19:03:49.148195 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:49.148283 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.188:35753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260811 19:03:49.149451 3511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:49.149136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.188:35753 (local address 127.0.0.1:33068)
0811 19:03:49.149278 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:49.149348 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.188:35753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 19:03:49.149637 3507 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.112.188:35753: Network error: Client connection negotiation failed: client connection to 127.29.112.188:35753: connect: Connection refused (error 111)
W20260811 19:03:49.180760 3519 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 19:03:49.181182 3519 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:03:49.181314 3519 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:03:49.181454 3519 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:03:49.188832 3519 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:03:49.188977 3519 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:03:49.189088 3519 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:03:49.189177 3519 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:03:49.189273 3519 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:03:49.199843 3519 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:40041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474615205622-30146-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474615205622-30146-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.29.112.190:36653,127.29.112.189:35303,127.29.112.188:35753
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.188.keytab
--principal=kudu/127.29.112.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.188:35753
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474615205622-30146-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.29.112.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474615205622-30146-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:03:49.201906 3519 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:03:49.203378 3519 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:03:49.216207 3528 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:03:49.216037 3530 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:03:49.216307 3532 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:03:49.217535 3519 server_base.cc:1034] running on GCE node
I20260811 19:03:49.218339 3519 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:03:49.220307 3519 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:03:49.221649 3519 hybrid_clock.cc:648] HybridClock initialized: now 1786475029221535 us; error 74 us; skew 500 ppm
I20260811 19:03:49.222105 3519 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:03:49 dist-test-slave-h6n0 krb5kdc[3261](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475029, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:03:49.225972 3519 init.cc:377] Logged in from keytab as kudu/127.29.112.188@KRBTEST.COM (short username kudu)
I20260811 19:03:49.227699 3519 webserver.cc:460] Webserver started at http://127.29.112.188:42857/ using document root <none> and password file <none>
I20260811 19:03:49.228464 3519 fs_manager.cc:359] Metadata directory not provided
I20260811 19:03:49.228567 3519 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:03:49.228850 3519 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:03:49.232463 3519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474615205622-30146-0/minicluster-data/master-2/data/instance:
uuid: "d6c7ffe696e34a23b96f62f57db165cd"
format_stamp: "Formatted at 2026-08-11 19:03:49 on dist-test-slave-h6n0"
server_key: "7e5c0165a8ea0c9ba9812b8c3551dc08"
server_key_iv: "b6a8bd65aa358eebe38f9a590088a4a9"
server_key_version: "encryptionkey@0"
I20260811 19:03:49.233539 3519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474615205622-30146-0/minicluster-data/master-2/wal/instance:
uuid: "d6c7ffe696e34a23b96f62f57db165cd"
format_stamp: "Formatted at 2026-08-11 19:03:49 on dist-test-slave-h6n0"
server_key: "7e5c0165a8ea0c9ba9812b8c3551dc08"
server_key_iv: "b6a8bd65aa358eebe38f9a590088a4a9"
server_key_version: "encryptionkey@0"
I20260811 19:03:49.240885 3519 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.000s sys 0.008s
I20260811 19:03:49.245627 3540 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:03:49.247735 3519 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260811 19:03:49.247962 3519 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474615205622-30146-0/minicluster-data/master-2/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474615205622-30146-0/minicluster-data/master-2/wal
uuid: "d6c7ffe696e34a23b96f62f57db165cd"
format_stamp: "Formatted at 2026-08-11 19:03:49 on dist-test-slave-h6n0"
server_key: "7e5c0165a8ea0c9ba9812b8c3551dc08"
server_key_iv: "b6a8bd65aa358eebe38f9a590088a4a9"
server_key_version: "encryptionkey@0"
I20260811 19:03:49.248260 3519 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474615205622-30146-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474615205622-30146-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474615205622-30146-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:03:49.261292 3398 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.112.188" port: 35753 } attempt: 4
I20260811 19:03:49.262708 3400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:49.262139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.188:35753 (local address 127.0.0.1:33070)
0811 19:03:49.262349 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:49.262457 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.188:35753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260811 19:03:49.264147 3400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:49.263729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.188:35753 (local address 127.0.0.1:33076)
0811 19:03:49.263917 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:49.264002 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.188:35753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
W20260811 19:03:49.264492 3398 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.112.188:35753: Network error: Client connection negotiation failed: client connection to 127.29.112.188:35753: connect: Connection refused (error 111)
I20260811 19:03:49.288437 3519 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:03:49.292784 3519 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:03:49.293068 3519 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:03:49.315032 3519 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.188:35753
I20260811 19:03:49.315056 3629 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.188:35753 every 8 connection(s)
I20260811 19:03:49.316437 3519 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474615205622-30146-0/minicluster-data/master-2/data/info.pb
I20260811 19:03:49.320210 3630 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:03:49.325250 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 3519
I20260811 19:03:49.325472 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474615205622-30146-0/minicluster-data/master-2/wal/instance
I20260811 19:03:49.325991 30146 external_mini_cluster.cc:1428] Setting key 54762b4f82c026b183ab01a61f7bf622
I20260811 19:03:49.327488 3630 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.112.190" port: 36653 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 19:03:49 dist-test-slave-h6n0 krb5kdc[3261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475029, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.188@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
Aug 11 19:03:49 dist-test-slave-h6n0 krb5kdc[3261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475027, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.188@KRBTEST.COM
I20260811 19:03:49.350461 3638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:49.330473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49648 (local address 127.29.112.190:36653)
0811 19:03:49.331943 (+ 1470us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:49.331949 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:49.331972 (+ 23us) server_negotiation.cc:407] Connection header received
0811 19:03:49.332122 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:49.332131 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:49.332228 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:49.332331 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:49.334274 (+ 1943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:49.335719 (+ 1445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:49.338194 (+ 2475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:49.338860 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:49.339022 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:49.343496 (+ 4474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:03:49.343500 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:03:49.343504 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:03:49.343574 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:03:49.346793 (+ 3219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:49.347745 (+ 952us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:49.347749 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:49.347751 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:49.347873 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:49.348318 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:49.348325 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:49.348328 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:49.348711 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:03:49.348836 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:49.349819 (+ 983us) server_negotiation.cc:299] Negotiation successful
0811 19:03:49.350029 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1352,"thread_start_us":1248,"threads_started":1}
I20260811 19:03:49.351281 3637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:49.330457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:36653 (local address 127.0.0.1:49648)
0811 19:03:49.331489 (+ 1032us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:49.331545 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 19:03:49.331894 (+ 349us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:03:49.332423 (+ 529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:03:49.332430 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:03:49.333351 (+ 921us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:03:49.333910 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:03:49.333939 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:03:49.335933 (+ 1994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:03:49.335939 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:03:49.337991 (+ 2052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:03:49.338008 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:03:49.339090 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:03:49.339096 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:03:49.339358 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:49.340174 (+ 816us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:03:49.340226 (+ 52us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:03:49.343300 (+ 3074us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:03:49.346926 (+ 3626us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:03:49.346936 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:03:49.346947 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:03:49.347630 (+ 683us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:03:49.347987 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:03:49.347994 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:03:49.347997 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:03:49.348204 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:03:49.348855 (+ 651us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:03:49.348863 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:03:49.349222 (+ 359us) client_negotiation.cc:769] Sending connection context
0811 19:03:49.349457 (+ 235us) client_negotiation.cc:240] Negotiation successful
0811 19:03:49.349918 (+ 461us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":693,"mutex_wait_us":41,"thread_start_us":466,"threads_started":1}
I20260811 19:03:49.351783 3636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:49.328121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33078 (local address 127.29.112.188:35753)
0811 19:03:49.329128 (+ 1007us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:49.329140 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:49.329181 (+ 41us) server_negotiation.cc:407] Connection header received
0811 19:03:49.331674 (+ 2493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:49.331706 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:49.332015 (+ 309us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:49.332258 (+ 243us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:49.334288 (+ 2030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:49.336194 (+ 1906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:49.338213 (+ 2019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:49.339100 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:49.339292 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:49.344522 (+ 5230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:03:49.344531 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:03:49.344542 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:03:49.344574 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:03:49.347910 (+ 3336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:49.348713 (+ 803us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:49.348727 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:49.348730 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:49.348897 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:49.349690 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:49.349695 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:49.349698 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:49.350116 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:03:49.350427 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:49.350998 (+ 571us) server_negotiation.cc:299] Negotiation successful
0811 19:03:49.351279 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":50,"server-negotiator.queue_time_us":801,"thread_start_us":574,"threads_started":1}
I20260811 19:03:49.353271 3630 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.112.189" port: 35303 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 19:03:49 dist-test-slave-h6n0 krb5kdc[3261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475029, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.188@KRBTEST.COM for kudu/127.29.112.189@KRBTEST.COM
Aug 11 19:03:49 dist-test-slave-h6n0 krb5kdc[3261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475027, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 19:03:49.371017 3637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:49.354115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.189:35303 (local address 127.0.0.1:45092)
0811 19:03:49.354272 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:49.354290 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 19:03:49.354381 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:03:49.355199 (+ 818us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:03:49.355204 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:03:49.355712 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:03:49.355905 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:03:49.355916 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:03:49.357412 (+ 1496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:03:49.357417 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:03:49.358781 (+ 1364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:03:49.358792 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:03:49.359720 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:03:49.359724 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:03:49.359893 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:49.360565 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:03:49.360588 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:03:49.362809 (+ 2221us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:03:49.367965 (+ 5156us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:03:49.367971 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:03:49.367975 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:03:49.368522 (+ 547us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:03:49.368950 (+ 428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:03:49.368957 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:03:49.368963 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:03:49.369147 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:03:49.369953 (+ 806us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:03:49.369958 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:03:49.370151 (+ 193us) client_negotiation.cc:769] Sending connection context
0811 19:03:49.370320 (+ 169us) client_negotiation.cc:240] Negotiation successful
0811 19:03:49.370655 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260811 19:03:49.371135 3641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:49.354399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45092 (local address 127.29.112.189:35303)
0811 19:03:49.354898 (+ 499us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:49.354902 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:49.354927 (+ 25us) server_negotiation.cc:407] Connection header received
0811 19:03:49.354975 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:49.354978 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:49.355041 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:49.355167 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:49.356101 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:49.357267 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:49.358906 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:49.359556 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:49.359684 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:49.363175 (+ 3491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:03:49.363192 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:03:49.363197 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:03:49.363244 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:03:49.367829 (+ 4585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:49.368671 (+ 842us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:49.368676 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:49.368679 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:49.368824 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:49.369305 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:49.369311 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:49.369314 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:49.369814 (+ 500us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:03:49.369960 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:49.370418 (+ 458us) server_negotiation.cc:299] Negotiation successful
0811 19:03:49.370692 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":399,"thread_start_us":305,"threads_started":1}
I20260811 19:03:49.371964 3630 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.112.188" port: 35753 } has no permanent_uuid. Determining permanent_uuid...
I20260811 19:03:49.374172 3638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:49.355334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49656 (local address 127.29.112.190:36653)
0811 19:03:49.355542 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:49.355548 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:49.355568 (+ 20us) server_negotiation.cc:407] Connection header received
0811 19:03:49.355694 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:49.355699 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:49.355782 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:49.355885 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:49.357412 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:49.358455 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:49.360129 (+ 1674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:49.360748 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:49.360890 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:49.367496 (+ 6606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:03:49.367501 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:03:49.367521 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:03:49.367554 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:03:49.370700 (+ 3146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:49.371527 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:49.371536 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:49.371538 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:49.371656 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:49.372236 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:49.372241 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:49.372244 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:49.372589 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:03:49.372872 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:49.373753 (+ 881us) server_negotiation.cc:299] Negotiation successful
0811 19:03:49.373921 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
Aug 11 19:03:49 dist-test-slave-h6n0 krb5kdc[3261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475029, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.188@KRBTEST.COM for kudu/127.29.112.188@KRBTEST.COM
Aug 11 19:03:49 dist-test-slave-h6n0 krb5kdc[3261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475027, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.189@KRBTEST.COM
I20260811 19:03:49.387811 3636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:49.373147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33082 (local address 127.29.112.188:35753)
0811 19:03:49.373330 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:49.373335 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:49.373353 (+ 18us) server_negotiation.cc:407] Connection header received
0811 19:03:49.373401 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:49.373406 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:49.373469 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:49.373567 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:49.374694 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:49.375579 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:49.377231 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:49.377648 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:49.377769 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:49.382353 (+ 4584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:03:49.382359 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:03:49.382364 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:03:49.382407 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:03:49.385067 (+ 2660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:49.385786 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:49.385793 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:49.385797 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:49.385947 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:49.386296 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:49.386303 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:49.386307 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:49.386635 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:03:49.386785 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:49.387136 (+ 351us) server_negotiation.cc:299] Negotiation successful
0811 19:03:49.387518 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":150,"server-negotiator.queue_time_us":57}
I20260811 19:03:49.387810 3637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:49.372843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.188:35753 (local address 127.0.0.1:33082)
0811 19:03:49.373006 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:49.373025 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 19:03:49.373123 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:03:49.373618 (+ 495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:03:49.373626 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:03:49.374033 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:03:49.374189 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:03:49.374197 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:03:49.375708 (+ 1511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:03:49.375713 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:03:49.377078 (+ 1365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:03:49.377096 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:03:49.377801 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:03:49.377807 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:03:49.377992 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:49.378631 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:03:49.378660 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:03:49.381920 (+ 3260us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:03:49.385210 (+ 3290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:03:49.385214 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:03:49.385216 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:03:49.385673 (+ 457us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:03:49.386054 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:03:49.386062 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:03:49.386066 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:03:49.386195 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:03:49.386741 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:03:49.386745 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:03:49.386896 (+ 151us) client_negotiation.cc:769] Sending connection context
0811 19:03:49.387053 (+ 157us) client_negotiation.cc:240] Negotiation successful
0811 19:03:49.387357 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53,"mutex_wait_us":1}
I20260811 19:03:49.392238 3641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:49.375761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45108 (local address 127.29.112.189:35303)
0811 19:03:49.375916 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:49.375920 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:49.375969 (+ 49us) server_negotiation.cc:407] Connection header received
0811 19:03:49.376146 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:49.376149 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:49.376200 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:49.376272 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:49.377813 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:49.378501 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:49.380423 (+ 1922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:49.381293 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:49.381406 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:49.386468 (+ 5062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:03:49.386475 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:03:49.386479 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:03:49.386515 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:03:49.389326 (+ 2811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:49.389988 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:49.389996 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:49.389999 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:49.390145 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:49.390510 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:49.390517 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:49.390520 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:49.390890 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:03:49.391170 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:49.391653 (+ 483us) server_negotiation.cc:299] Negotiation successful
0811 19:03:49.391905 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 19:03:49.393771 3630 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6c7ffe696e34a23b96f62f57db165cd: Bootstrap starting.
I20260811 19:03:49.397071 30146 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 19:03:49.397262 3630 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d6c7ffe696e34a23b96f62f57db165cd: Neither blocks nor log segments found. Creating new log.
I20260811 19:03:49.398849 3630 log.cc:824] T 00000000000000000000000000000000 P d6c7ffe696e34a23b96f62f57db165cd: Log is configured to *not* fsync() on all Append() calls
I20260811 19:03:49.402416 3630 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6c7ffe696e34a23b96f62f57db165cd: No bootstrap required, opened a new log
I20260811 19:03:49.399839 3356 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "1217462373bb43d08e40cfbba99b8378"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49656
I20260811 19:03:49.410805 3630 raft_consensus.cc:348] T 00000000000000000000000000000000 P d6c7ffe696e34a23b96f62f57db165cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1217462373bb43d08e40cfbba99b8378" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 36653 } } peers { permanent_uuid: "66b539a9a527407181be314b47238b2f" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 35303 } } peers { permanent_uuid: "d6c7ffe696e34a23b96f62f57db165cd" member_type: VOTER last_known_addr { host: "127.29.112.188" port: 35753 } }
I20260811 19:03:49.411217 3630 raft_consensus.cc:729] T 00000000000000000000000000000000 P d6c7ffe696e34a23b96f62f57db165cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6c7ffe696e34a23b96f62f57db165cd, State: Initialized, Role: FOLLOWER
I20260811 19:03:49.411782 3630 consensus_queue.cc:260] T 00000000000000000000000000000000 P d6c7ffe696e34a23b96f62f57db165cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1217462373bb43d08e40cfbba99b8378" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 36653 } } peers { permanent_uuid: "66b539a9a527407181be314b47238b2f" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 35303 } } peers { permanent_uuid: "d6c7ffe696e34a23b96f62f57db165cd" member_type: VOTER last_known_addr { host: "127.29.112.188" port: 35753 } }
I20260811 19:03:49.412801 3645 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6c7ffe696e34a23b96f62f57db165cd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1217462373bb43d08e40cfbba99b8378" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 36653 } } peers { permanent_uuid: "66b539a9a527407181be314b47238b2f" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 35303 } } peers { permanent_uuid: "d6c7ffe696e34a23b96f62f57db165cd" member_type: VOTER last_known_addr { host: "127.29.112.188" port: 35753 } } }
I20260811 19:03:49.413153 3645 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6c7ffe696e34a23b96f62f57db165cd [sys.catalog]: This master's current role is: FOLLOWER
I20260811 19:03:49.414083 3630 sys_catalog.cc:564] T 00000000000000000000000000000000 P d6c7ffe696e34a23b96f62f57db165cd [sys.catalog]: configured and running, proceeding with master startup.
W20260811 19:03:49.424069 3667 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d6c7ffe696e34a23b96f62f57db165cd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 19:03:49.424204 3667 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 19:03:49.422160 3519 master_runner.cc:186] Error getting master registration for 127.29.112.190:36653: OK, instance_id { permanent_uuid: "1217462373bb43d08e40cfbba99b8378" instance_seqno: 1786475028104252 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 19:03:49.433867 3519 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 19:03:49.448869 3507 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.112.188" port: 35753 } attempt: 5
Aug 11 19:03:49 dist-test-slave-h6n0 krb5kdc[3261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475028, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.189@KRBTEST.COM for kudu/127.29.112.188@KRBTEST.COM
I20260811 19:03:49.466647 3636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:49.449876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33096 (local address 127.29.112.188:35753)
0811 19:03:49.450312 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:49.450319 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:49.450337 (+ 18us) server_negotiation.cc:407] Connection header received
0811 19:03:49.450398 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:49.450404 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:49.450483 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:49.450610 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:49.451757 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:49.452960 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:49.454724 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:49.455320 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:49.455443 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:49.459278 (+ 3835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:03:49.459284 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:03:49.459295 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:03:49.459326 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:03:49.463684 (+ 4358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:49.464340 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:49.464347 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:49.464350 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:49.464482 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:49.464898 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:49.464903 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:49.464907 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:49.465414 (+ 507us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:03:49.465575 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:49.466020 (+ 445us) server_negotiation.cc:299] Negotiation successful
0811 19:03:49.466241 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288}
I20260811 19:03:49.466642 3511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:49.449667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.188:35753 (local address 127.0.0.1:33096)
0811 19:03:49.449892 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:49.449917 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 19:03:49.450064 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:03:49.450803 (+ 739us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:03:49.450808 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:03:49.451393 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:03:49.451603 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:03:49.451617 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:03:49.453118 (+ 1501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:03:49.453124 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:03:49.454577 (+ 1453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:03:49.454592 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:03:49.455489 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:03:49.455495 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:03:49.455742 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:49.456452 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:03:49.456475 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:03:49.459095 (+ 2620us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:03:49.463821 (+ 4726us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:03:49.463827 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:03:49.463831 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:03:49.464221 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:03:49.464606 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:03:49.464611 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:03:49.464614 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:03:49.464787 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:03:49.465544 (+ 757us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:03:49.465549 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:03:49.465709 (+ 160us) client_negotiation.cc:769] Sending connection context
0811 19:03:49.465894 (+ 185us) client_negotiation.cc:240] Negotiation successful
0811 19:03:49.466209 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74}
I20260811 19:03:49.472954 3507 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66b539a9a527407181be314b47238b2f: Bootstrap starting.
I20260811 19:03:49.477119 3507 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 66b539a9a527407181be314b47238b2f: Neither blocks nor log segments found. Creating new log.
I20260811 19:03:49.478395 3507 log.cc:824] T 00000000000000000000000000000000 P 66b539a9a527407181be314b47238b2f: Log is configured to *not* fsync() on all Append() calls
I20260811 19:03:49.481778 3507 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66b539a9a527407181be314b47238b2f: No bootstrap required, opened a new log
I20260811 19:03:49.493021 3507 raft_consensus.cc:348] T 00000000000000000000000000000000 P 66b539a9a527407181be314b47238b2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1217462373bb43d08e40cfbba99b8378" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 36653 } } peers { permanent_uuid: "66b539a9a527407181be314b47238b2f" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 35303 } } peers { permanent_uuid: "d6c7ffe696e34a23b96f62f57db165cd" member_type: VOTER last_known_addr { host: "127.29.112.188" port: 35753 } }
I20260811 19:03:49.493436 3507 raft_consensus.cc:729] T 00000000000000000000000000000000 P 66b539a9a527407181be314b47238b2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66b539a9a527407181be314b47238b2f, State: Initialized, Role: FOLLOWER
I20260811 19:03:49.493956 3507 consensus_queue.cc:260] T 00000000000000000000000000000000 P 66b539a9a527407181be314b47238b2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1217462373bb43d08e40cfbba99b8378" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 36653 } } peers { permanent_uuid: "66b539a9a527407181be314b47238b2f" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 35303 } } peers { permanent_uuid: "d6c7ffe696e34a23b96f62f57db165cd" member_type: VOTER last_known_addr { host: "127.29.112.188" port: 35753 } }
I20260811 19:03:49.495132 3675 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66b539a9a527407181be314b47238b2f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1217462373bb43d08e40cfbba99b8378" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 36653 } } peers { permanent_uuid: "66b539a9a527407181be314b47238b2f" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 35303 } } peers { permanent_uuid: "d6c7ffe696e34a23b96f62f57db165cd" member_type: VOTER last_known_addr { host: "127.29.112.188" port: 35753 } } }
I20260811 19:03:49.495608 3675 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66b539a9a527407181be314b47238b2f [sys.catalog]: This master's current role is: FOLLOWER
I20260811 19:03:49.496285 3507 sys_catalog.cc:564] T 00000000000000000000000000000000 P 66b539a9a527407181be314b47238b2f [sys.catalog]: configured and running, proceeding with master startup.
W20260811 19:03:49.507413 3690 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 66b539a9a527407181be314b47238b2f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 19:03:49.507658 3690 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 19:03:49.504024 3408 master_runner.cc:186] Error getting master registration for 127.29.112.190:36653: OK, instance_id { permanent_uuid: "1217462373bb43d08e40cfbba99b8378" instance_seqno: 1786475028104252 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 19:03:49.516789 3408 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 19:03:49.541750 3398 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.112.188" port: 35753 } attempt: 5
W20260811 19:03:49.545579 3549 tablet.cc:2369] T 00000000000000000000000000000000 P d6c7ffe696e34a23b96f62f57db165cd: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 11 19:03:49 dist-test-slave-h6n0 krb5kdc[3261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475028, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.190@KRBTEST.COM for kudu/127.29.112.188@KRBTEST.COM
I20260811 19:03:49.559303 3636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:49.542720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33108 (local address 127.29.112.188:35753)
0811 19:03:49.542916 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:49.542923 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:49.543000 (+ 77us) server_negotiation.cc:407] Connection header received
0811 19:03:49.543214 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:49.543220 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:49.543303 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:49.543405 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:49.544583 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:49.545964 (+ 1381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:49.548329 (+ 2365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:49.548780 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:49.548917 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:49.553055 (+ 4138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:03:49.553062 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:03:49.553067 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:03:49.553111 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:03:49.556380 (+ 3269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:49.557194 (+ 814us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:49.557199 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:49.557203 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:49.557351 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:49.557799 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:49.557804 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:49.557812 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:49.558079 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:03:49.558246 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:49.558682 (+ 436us) server_negotiation.cc:299] Negotiation successful
0811 19:03:49.558932 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 19:03:49.559297 3400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:49.542627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.188:35753 (local address 127.0.0.1:33108)
0811 19:03:49.542918 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:49.542940 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 19:03:49.543072 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:03:49.543600 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:03:49.543607 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:03:49.544208 (+ 601us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:03:49.544423 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:03:49.544437 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:03:49.546600 (+ 2163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:03:49.546608 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:03:49.548170 (+ 1562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:03:49.548185 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:03:49.548932 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:03:49.548938 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:03:49.549147 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:49.549922 (+ 775us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:03:49.549948 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:03:49.552843 (+ 2895us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:03:49.556544 (+ 3701us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:03:49.556550 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:03:49.556556 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:03:49.557068 (+ 512us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:03:49.557491 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:03:49.557498 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:03:49.557503 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:03:49.557683 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:03:49.558202 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:03:49.558207 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:03:49.558412 (+ 205us) client_negotiation.cc:769] Sending connection context
0811 19:03:49.558584 (+ 172us) client_negotiation.cc:240] Negotiation successful
0811 19:03:49.558862 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159}
I20260811 19:03:49.565794 3398 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1217462373bb43d08e40cfbba99b8378: Bootstrap starting.
I20260811 19:03:49.569895 3398 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1217462373bb43d08e40cfbba99b8378: Neither blocks nor log segments found. Creating new log.
I20260811 19:03:49.571206 3398 log.cc:824] T 00000000000000000000000000000000 P 1217462373bb43d08e40cfbba99b8378: Log is configured to *not* fsync() on all Append() calls
I20260811 19:03:49.574817 3398 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1217462373bb43d08e40cfbba99b8378: No bootstrap required, opened a new log
I20260811 19:03:49.583480 3398 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1217462373bb43d08e40cfbba99b8378 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1217462373bb43d08e40cfbba99b8378" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 36653 } } peers { permanent_uuid: "66b539a9a527407181be314b47238b2f" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 35303 } } peers { permanent_uuid: "d6c7ffe696e34a23b96f62f57db165cd" member_type: VOTER last_known_addr { host: "127.29.112.188" port: 35753 } }
I20260811 19:03:49.584185 3398 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1217462373bb43d08e40cfbba99b8378 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1217462373bb43d08e40cfbba99b8378, State: Initialized, Role: FOLLOWER
I20260811 19:03:49.584904 3398 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1217462373bb43d08e40cfbba99b8378 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1217462373bb43d08e40cfbba99b8378" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 36653 } } peers { permanent_uuid: "66b539a9a527407181be314b47238b2f" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 35303 } } peers { permanent_uuid: "d6c7ffe696e34a23b96f62f57db165cd" member_type: VOTER last_known_addr { host: "127.29.112.188" port: 35753 } }
I20260811 19:03:49.587076 3695 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1217462373bb43d08e40cfbba99b8378 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1217462373bb43d08e40cfbba99b8378" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 36653 } } peers { permanent_uuid: "66b539a9a527407181be314b47238b2f" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 35303 } } peers { permanent_uuid: "d6c7ffe696e34a23b96f62f57db165cd" member_type: VOTER last_known_addr { host: "127.29.112.188" port: 35753 } } }
I20260811 19:03:49.588105 3695 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1217462373bb43d08e40cfbba99b8378 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 19:03:49.588341 3398 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1217462373bb43d08e40cfbba99b8378 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:03:49.588423 3356 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1217462373bb43d08e40cfbba99b8378 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260811 19:03:49.588642 3356 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1217462373bb43d08e40cfbba99b8378 [term 0 FOLLOWER]: Advancing to term 1
W20260811 19:03:49.592878 3322 tablet.cc:2369] T 00000000000000000000000000000000 P 1217462373bb43d08e40cfbba99b8378: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 19:03:49.593644 3356 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1217462373bb43d08e40cfbba99b8378 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1217462373bb43d08e40cfbba99b8378" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 36653 } } peers { permanent_uuid: "66b539a9a527407181be314b47238b2f" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 35303 } } peers { permanent_uuid: "d6c7ffe696e34a23b96f62f57db165cd" member_type: VOTER last_known_addr { host: "127.29.112.188" port: 35753 } }
I20260811 19:03:49.596979 3465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1217462373bb43d08e40cfbba99b8378" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "66b539a9a527407181be314b47238b2f"
I20260811 19:03:49.597569 3465 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 66b539a9a527407181be314b47238b2f [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:03:49.597592 3582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1217462373bb43d08e40cfbba99b8378" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "d6c7ffe696e34a23b96f62f57db165cd"
I20260811 19:03:49.598172 3582 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d6c7ffe696e34a23b96f62f57db165cd [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:03:49.601554 3465 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 66b539a9a527407181be314b47238b2f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1217462373bb43d08e40cfbba99b8378 in term 1.
I20260811 19:03:49.602459 3312 leader_election.cc:302] T 00000000000000000000000000000000 P 1217462373bb43d08e40cfbba99b8378 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1217462373bb43d08e40cfbba99b8378, 66b539a9a527407181be314b47238b2f; no voters:
I20260811 19:03:49.604113 3698 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1217462373bb43d08e40cfbba99b8378 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:03:49.604422 3582 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d6c7ffe696e34a23b96f62f57db165cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1217462373bb43d08e40cfbba99b8378 in term 1.
I20260811 19:03:49.605342 3356 leader_election.cc:288] T 00000000000000000000000000000000 P 1217462373bb43d08e40cfbba99b8378 [CANDIDATE]: Term 1 election: Requested vote from peers 66b539a9a527407181be314b47238b2f (127.29.112.189:35303), d6c7ffe696e34a23b96f62f57db165cd (127.29.112.188:35753)
I20260811 19:03:49.620352 3698 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1217462373bb43d08e40cfbba99b8378 [term 1 LEADER]: Becoming Leader. State: Replica: 1217462373bb43d08e40cfbba99b8378, State: Running, Role: LEADER
I20260811 19:03:49.621469 3698 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1217462373bb43d08e40cfbba99b8378 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1217462373bb43d08e40cfbba99b8378" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 36653 } } peers { permanent_uuid: "66b539a9a527407181be314b47238b2f" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 35303 } } peers { permanent_uuid: "d6c7ffe696e34a23b96f62f57db165cd" member_type: VOTER last_known_addr { host: "127.29.112.188" port: 35753 } }
I20260811 19:03:49.636194 3695 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1217462373bb43d08e40cfbba99b8378 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1217462373bb43d08e40cfbba99b8378. Latest consensus state: current_term: 1 leader_uuid: "1217462373bb43d08e40cfbba99b8378" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1217462373bb43d08e40cfbba99b8378" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 36653 } } peers { permanent_uuid: "66b539a9a527407181be314b47238b2f" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 35303 } } peers { permanent_uuid: "d6c7ffe696e34a23b96f62f57db165cd" member_type: VOTER last_known_addr { host: "127.29.112.188" port: 35753 } } }
I20260811 19:03:49.636575 3695 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1217462373bb43d08e40cfbba99b8378 [sys.catalog]: This master's current role is: LEADER
I20260811 19:03:49.659945 3717 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:03:49.685679 3638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:49.634972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49668 (local address 127.29.112.190:36653)
0811 19:03:49.637060 (+ 2088us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:49.637068 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:49.654179 (+ 17111us) server_negotiation.cc:407] Connection header received
0811 19:03:49.654551 (+ 372us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:49.654563 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:49.654677 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:49.654771 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:49.660660 (+ 5889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:49.661954 (+ 1294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:49.666715 (+ 4761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:49.667588 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:49.667692 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:49.675957 (+ 8265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:03:49.675967 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:03:49.675975 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:03:49.676026 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:03:49.680163 (+ 4137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:49.681912 (+ 1749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:49.681918 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:49.681922 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:49.682110 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:49.683046 (+ 936us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:49.683055 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:49.683061 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:49.683514 (+ 453us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:03:49.683829 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:49.684914 (+ 1085us) server_negotiation.cc:299] Negotiation successful
0811 19:03:49.685196 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1949}
I20260811 19:03:49.691131 3717 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260811 19:03:49.694466 3439 tablet.cc:2369] T 00000000000000000000000000000000 P 66b539a9a527407181be314b47238b2f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 19:03:49.740489 3582 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d6c7ffe696e34a23b96f62f57db165cd [term 1 FOLLOWER]: Refusing update from remote peer 1217462373bb43d08e40cfbba99b8378: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:03:49.743280 3465 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 66b539a9a527407181be314b47238b2f [term 1 FOLLOWER]: Refusing update from remote peer 1217462373bb43d08e40cfbba99b8378: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:03:49.745337 3695 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1217462373bb43d08e40cfbba99b8378 [LEADER]: Connected to new peer: Peer: permanent_uuid: "66b539a9a527407181be314b47238b2f" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 35303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:03:49.746124 3695 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1217462373bb43d08e40cfbba99b8378 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6c7ffe696e34a23b96f62f57db165cd" member_type: VOTER last_known_addr { host: "127.29.112.188" port: 35753 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:03:49.772549 3641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:49.663989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45112 (local address 127.29.112.189:35303)
0811 19:03:49.664183 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:49.664189 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:49.667551 (+ 3362us) server_negotiation.cc:407] Connection header received
0811 19:03:49.674421 (+ 6870us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:49.674427 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:49.674571 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:49.674677 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:49.681478 (+ 6801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:49.682967 (+ 1489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:49.696950 (+ 13983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:49.697942 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:49.698125 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:49.727706 (+ 29581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:03:49.727715 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:03:49.727721 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:03:49.727787 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:03:49.736379 (+ 8592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:49.759165 (+ 22786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:49.759172 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:49.759176 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:49.759396 (+ 220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:49.762261 (+ 2865us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:49.762268 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:49.762271 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:49.762827 (+ 556us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:03:49.763135 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:49.767311 (+ 4176us) server_negotiation.cc:299] Negotiation successful
0811 19:03:49.767605 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 19:03:49.773840 3636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:49.653894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33122 (local address 127.29.112.188:35753)
0811 19:03:49.654095 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:49.654105 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:49.663248 (+ 9143us) server_negotiation.cc:407] Connection header received
0811 19:03:49.664737 (+ 1489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:49.664746 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:49.664909 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:49.665004 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:49.677826 (+ 12822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:49.679202 (+ 1376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:49.691582 (+ 12380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:49.693011 (+ 1429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:49.693216 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:49.729275 (+ 36059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:03:49.729288 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:03:49.729296 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:03:49.729374 (+ 78us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:03:49.758898 (+ 29524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:49.764545 (+ 5647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:49.764698 (+ 153us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:49.764704 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:49.764903 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:49.765967 (+ 1064us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:49.765972 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:49.765976 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:49.766806 (+ 830us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:03:49.770036 (+ 3230us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:49.770210 (+ 174us) server_negotiation.cc:299] Negotiation successful
0811 19:03:49.770514 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 19:03:49.779954 3675 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66b539a9a527407181be314b47238b2f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1217462373bb43d08e40cfbba99b8378. Latest consensus state: current_term: 1 leader_uuid: "1217462373bb43d08e40cfbba99b8378" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1217462373bb43d08e40cfbba99b8378" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 36653 } } peers { permanent_uuid: "66b539a9a527407181be314b47238b2f" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 35303 } } peers { permanent_uuid: "d6c7ffe696e34a23b96f62f57db165cd" member_type: VOTER last_known_addr { host: "127.29.112.188" port: 35753 } } }
I20260811 19:03:49.780309 3675 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66b539a9a527407181be314b47238b2f [sys.catalog]: This master's current role is: FOLLOWER
I20260811 19:03:49.799903 3695 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1217462373bb43d08e40cfbba99b8378 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1217462373bb43d08e40cfbba99b8378" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1217462373bb43d08e40cfbba99b8378" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 36653 } } peers { permanent_uuid: "66b539a9a527407181be314b47238b2f" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 35303 } } peers { permanent_uuid: "d6c7ffe696e34a23b96f62f57db165cd" member_type: VOTER last_known_addr { host: "127.29.112.188" port: 35753 } } }
I20260811 19:03:49.800318 3695 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1217462373bb43d08e40cfbba99b8378 [sys.catalog]: This master's current role is: LEADER
I20260811 19:03:49.809058 3717 catalog_manager.cc:1347] Generated new cluster ID: d67e409946f043d690439977311fdc19
I20260811 19:03:49.809197 3717 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:03:49.820367 3645 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6c7ffe696e34a23b96f62f57db165cd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1217462373bb43d08e40cfbba99b8378. Latest consensus state: current_term: 1 leader_uuid: "1217462373bb43d08e40cfbba99b8378" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1217462373bb43d08e40cfbba99b8378" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 36653 } } peers { permanent_uuid: "66b539a9a527407181be314b47238b2f" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 35303 } } peers { permanent_uuid: "d6c7ffe696e34a23b96f62f57db165cd" member_type: VOTER last_known_addr { host: "127.29.112.188" port: 35753 } } }
I20260811 19:03:49.820731 3645 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6c7ffe696e34a23b96f62f57db165cd [sys.catalog]: This master's current role is: FOLLOWER
I20260811 19:03:49.822146 3675 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66b539a9a527407181be314b47238b2f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1217462373bb43d08e40cfbba99b8378" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1217462373bb43d08e40cfbba99b8378" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 36653 } } peers { permanent_uuid: "66b539a9a527407181be314b47238b2f" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 35303 } } peers { permanent_uuid: "d6c7ffe696e34a23b96f62f57db165cd" member_type: VOTER last_known_addr { host: "127.29.112.188" port: 35753 } } }
I20260811 19:03:49.822435 3675 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66b539a9a527407181be314b47238b2f [sys.catalog]: This master's current role is: FOLLOWER
I20260811 19:03:49.832921 3717 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:03:49.834885 3717 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:03:49.855656 3695 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1217462373bb43d08e40cfbba99b8378 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1217462373bb43d08e40cfbba99b8378" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1217462373bb43d08e40cfbba99b8378" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 36653 } } peers { permanent_uuid: "66b539a9a527407181be314b47238b2f" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 35303 } } peers { permanent_uuid: "d6c7ffe696e34a23b96f62f57db165cd" member_type: VOTER last_known_addr { host: "127.29.112.188" port: 35753 } } }
I20260811 19:03:49.855985 3695 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1217462373bb43d08e40cfbba99b8378 [sys.catalog]: This master's current role is: LEADER
I20260811 19:03:49.858392 3734 mvcc.cc:205] Tried to move back new op lower bound from 7317401721805500416 to 7317401721348825088. Current Snapshot: MvccSnapshot[applied={T|T < 7317401721805500416}]
I20260811 19:03:49.870151 3717 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1217462373bb43d08e40cfbba99b8378: Generated new TSK 0
I20260811 19:03:49.871330 3717 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 19:03:49.876369 3645 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6c7ffe696e34a23b96f62f57db165cd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1217462373bb43d08e40cfbba99b8378" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1217462373bb43d08e40cfbba99b8378" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 36653 } } peers { permanent_uuid: "66b539a9a527407181be314b47238b2f" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 35303 } } peers { permanent_uuid: "d6c7ffe696e34a23b96f62f57db165cd" member_type: VOTER last_known_addr { host: "127.29.112.188" port: 35753 } } }
I20260811 19:03:49.876637 3645 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6c7ffe696e34a23b96f62f57db165cd [sys.catalog]: This master's current role is: FOLLOWER
W20260811 19:03:50.021263 3638 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:03:50.008275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49678 (local address 127.29.112.190:36653)
0811 19:03:50.008746 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:50.008752 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:50.018608 (+ 9856us) server_negotiation.cc:407] Connection header received
0811 19:03:50.019189 (+ 581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:50.019196 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:50.019299 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:50.019383 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:50.020738 (+ 1355us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49678: BlockingRecv error: recv got EOF from 127.0.0.1:49678 (error 108)
Metrics: {"server-negotiator.queue_time_us":318}
W20260811 19:03:50.024506 3636 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:03:50.008521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33124 (local address 127.29.112.188:35753)
0811 19:03:50.008703 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:50.008710 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:50.022164 (+ 13454us) server_negotiation.cc:407] Connection header received
0811 19:03:50.022363 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:50.022373 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:50.022483 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:50.022618 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:50.024247 (+ 1629us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33124: BlockingRecv error: recv got EOF from 127.0.0.1:33124 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260811 19:03:50.025502 3641 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:03:50.023188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45128 (local address 127.29.112.189:35303)
0811 19:03:50.023401 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:50.023410 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:50.023572 (+ 162us) server_negotiation.cc:407] Connection header received
0811 19:03:50.023778 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:50.023784 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:50.023870 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:50.023955 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:50.025278 (+ 1323us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45128: BlockingRecv error: recv got EOF from 127.0.0.1:45128 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260811 19:03:50.032702 3641 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:03:50.030184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45140 (local address 127.29.112.189:35303)
0811 19:03:50.030409 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:50.030416 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:50.030976 (+ 560us) server_negotiation.cc:407] Connection header received
0811 19:03:50.031212 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:50.031218 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:50.031361 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:50.031502 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:50.032401 (+ 899us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45140: BlockingRecv error: recv got EOF from 127.0.0.1:45140 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260811 19:03:50.044698 3641 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:03:50.038512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45142 (local address 127.29.112.189:35303)
0811 19:03:50.038727 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:50.038733 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:50.043123 (+ 4390us) server_negotiation.cc:407] Connection header received
0811 19:03:50.043328 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:50.043335 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:50.043434 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:50.043551 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:50.044443 (+ 892us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45142: BlockingRecv error: recv got EOF from 127.0.0.1:45142 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
W20260811 19:03:50.053865 3641 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:03:50.049941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45154 (local address 127.29.112.189:35303)
0811 19:03:50.050113 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:50.050123 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:50.052370 (+ 2247us) server_negotiation.cc:407] Connection header received
0811 19:03:50.052601 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:50.052607 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:50.052698 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:50.052808 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:50.053601 (+ 793us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45154: BlockingRecv error: recv got EOF from 127.0.0.1:45154 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260811 19:03:50.064286 3641 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:03:50.061932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45158 (local address 127.29.112.189:35303)
0811 19:03:50.062166 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:50.062172 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:50.062741 (+ 569us) server_negotiation.cc:407] Connection header received
0811 19:03:50.062955 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:50.062961 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:50.063063 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:50.063176 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:50.064004 (+ 828us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45158: BlockingRecv error: recv got EOF from 127.0.0.1:45158 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260811 19:03:50.078837 3641 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:03:50.076355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45172 (local address 127.29.112.189:35303)
0811 19:03:50.076617 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:50.076622 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:50.077170 (+ 548us) server_negotiation.cc:407] Connection header received
0811 19:03:50.077410 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:50.077417 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:50.077507 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:50.077614 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:50.078576 (+ 962us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45172: BlockingRecv error: recv got EOF from 127.0.0.1:45172 (error 108)
Metrics: {"server-negotiator.queue_time_us":123}
W20260811 19:03:50.100857 3641 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:03:50.098499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45178 (local address 127.29.112.189:35303)
0811 19:03:50.098708 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:50.098714 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:50.099281 (+ 567us) server_negotiation.cc:407] Connection header received
0811 19:03:50.099486 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:50.099492 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:50.099604 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:50.099720 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:50.100590 (+ 870us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45178: BlockingRecv error: recv got EOF from 127.0.0.1:45178 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
W20260811 19:03:50.139269 3641 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:03:50.137038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45180 (local address 127.29.112.189:35303)
0811 19:03:50.137266 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:50.137274 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:50.137662 (+ 388us) server_negotiation.cc:407] Connection header received
0811 19:03:50.137851 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:50.137858 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:50.137981 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:50.138119 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:50.138983 (+ 864us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45180: BlockingRecv error: recv got EOF from 127.0.0.1:45180 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
W20260811 19:03:50.208715 3641 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:03:50.206703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45192 (local address 127.29.112.189:35303)
0811 19:03:50.206933 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:50.206940 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:50.207247 (+ 307us) server_negotiation.cc:407] Connection header received
0811 19:03:50.207471 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:50.207481 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:50.207554 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:50.207660 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:50.208466 (+ 806us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45192: BlockingRecv error: recv got EOF from 127.0.0.1:45192 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
W20260811 19:03:50.346200 3641 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:03:50.343997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45196 (local address 127.29.112.189:35303)
0811 19:03:50.344187 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:50.344192 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:50.344538 (+ 346us) server_negotiation.cc:407] Connection header received
0811 19:03:50.344775 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:50.344780 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:50.344893 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:50.345026 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:50.346005 (+ 979us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45196: BlockingRecv error: recv got EOF from 127.0.0.1:45196 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 19:03:50.426919 3667 catalog_manager.cc:1259] Loaded cluster ID: d67e409946f043d690439977311fdc19
I20260811 19:03:50.427065 3667 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d6c7ffe696e34a23b96f62f57db165cd: loading cluster ID for follower catalog manager: success
I20260811 19:03:50.430160 3667 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d6c7ffe696e34a23b96f62f57db165cd: acquiring CA information for follower catalog manager: success
I20260811 19:03:50.432327 3667 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d6c7ffe696e34a23b96f62f57db165cd: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 19:03:50.509889 3690 catalog_manager.cc:1259] Loaded cluster ID: d67e409946f043d690439977311fdc19
I20260811 19:03:50.510011 3690 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 66b539a9a527407181be314b47238b2f: loading cluster ID for follower catalog manager: success
I20260811 19:03:50.512789 3690 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 66b539a9a527407181be314b47238b2f: acquiring CA information for follower catalog manager: success
I20260811 19:03:50.514750 3690 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 66b539a9a527407181be314b47238b2f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 19:03:50.614193 3641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:50.608465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45210 (local address 127.29.112.189:35303)
0811 19:03:50.608723 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:50.608731 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:50.608947 (+ 216us) server_negotiation.cc:407] Connection header received
0811 19:03:50.609195 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:50.609200 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:50.609367 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:50.609497 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:03:50.609945 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:50.610665 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:50.611975 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:50.612397 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:50.612533 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:50.613007 (+ 474us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:03:50.613506 (+ 499us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:03:50.613678 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:50.613732 (+ 54us) server_negotiation.cc:299] Negotiation successful
0811 19:03:50.613815 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260811 19:03:50.625905 3861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:50.619963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33136 (local address 127.29.112.188:35753)
0811 19:03:50.620416 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:50.620421 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:50.620525 (+ 104us) server_negotiation.cc:407] Connection header received
0811 19:03:50.620697 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:50.620701 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:50.620860 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:50.620997 (+ 137us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:03:50.621342 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:50.622046 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:50.623352 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:50.623723 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:50.623835 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:50.624334 (+ 499us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:03:50.624843 (+ 509us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:03:50.624964 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:50.625441 (+ 477us) server_negotiation.cc:299] Negotiation successful
0811 19:03:50.625563 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":248,"threads_started":1}
I20260811 19:03:50.643270 3861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:50.633132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33150 (local address 127.29.112.188:35753)
0811 19:03:50.633405 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:50.633414 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:50.635611 (+ 2197us) server_negotiation.cc:407] Connection header received
0811 19:03:50.635917 (+ 306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:50.635922 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:50.636007 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:50.636094 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:03:50.636672 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:50.637850 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:50.640449 (+ 2599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:50.640966 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:50.641109 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:50.641590 (+ 481us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:03:50.641729 (+ 139us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:03:50.641824 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:50.642588 (+ 764us) server_negotiation.cc:299] Negotiation successful
0811 19:03:50.642680 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260811 19:03:50.643787 3879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:50.633133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49692 (local address 127.29.112.190:36653)
0811 19:03:50.634997 (+ 1864us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:50.635003 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:50.635851 (+ 848us) server_negotiation.cc:407] Connection header received
0811 19:03:50.636192 (+ 341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:50.636201 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:50.636478 (+ 277us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:50.636681 (+ 203us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:03:50.637234 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:50.638212 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:50.640413 (+ 2201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:50.641377 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:50.641533 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:50.642310 (+ 777us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:03:50.642836 (+ 526us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:03:50.642975 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:50.643379 (+ 404us) server_negotiation.cc:299] Negotiation successful
0811 19:03:50.643475 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1706,"thread_start_us":1575,"threads_started":1}
I20260811 19:03:50.645220 3641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:50.635530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45212 (local address 127.29.112.189:35303)
0811 19:03:50.635710 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:50.635715 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:50.637720 (+ 2005us) server_negotiation.cc:407] Connection header received
0811 19:03:50.637868 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:50.637882 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:50.637965 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:50.638050 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:03:50.638914 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:50.640013 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:50.642885 (+ 2872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:50.643487 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:50.643613 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:50.644427 (+ 814us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:03:50.644551 (+ 124us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:03:50.644648 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:50.644853 (+ 205us) server_negotiation.cc:299] Negotiation successful
0811 19:03:50.644924 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 19:03:50.649418 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 3287
I20260811 19:03:50.650758 3387 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:03:50.764628 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 3287
I20260811 19:03:50.792536 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 3408
I20260811 19:03:50.793638 3497 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:03:50.906374 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 3408
I20260811 19:03:50.936698 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 3519
I20260811 19:03:50.938385 3622 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:03:51.057212 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 3519
2026-08-11T19:03:51Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (5196 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474615205622-30146-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:03:51 dist-test-slave-h6n0 krb5kdc[3889](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:03:51 dist-test-slave-h6n0 krb5kdc[3889](info): set up 2 sockets
Aug 11 19:03:51 dist-test-slave-h6n0 krb5kdc[3889](info): commencing operation
krb5kdc: starting...
W20260811 19:03:52.489606 30146 mini_kdc.cc:121] Time spent starting KDC: real 1.374s user 0.000s sys 0.049s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474615205622-30146-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474615205622-30146-0/krb5kdc/test-user.keytab.
Aug 11 19:03:52 dist-test-slave-h6n0 krb5kdc[3889](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475032, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:03:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:03:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.112.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab.
Entry for principal kudu/127.29.112.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab.
I20260811 19:03:52.841554 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474615205622-30146-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474615205622-30146-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474615205622-30146-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.190:33515
--webserver_interface=127.29.112.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab
--principal=kudu/127.29.112.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:33881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.190:33515,127.29.112.189:34961,127.29.112.188:44625
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:03:53.197064 3906 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 19:03:53.197464 3906 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:03:53.197592 3906 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:03:53.197726 3906 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:03:53.205067 3906 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:03:53.205199 3906 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:03:53.205335 3906 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:03:53.205430 3906 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:03:53.205539 3906 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:03:53.215638 3906 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:33881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474615205622-30146-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474615205622-30146-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.112.190:33515,127.29.112.189:34961,127.29.112.188:44625
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.190.keytab
--principal=kudu/127.29.112.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.190:33515
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.112.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474615205622-30146-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:03:53.216967 3906 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:03:53.218291 3906 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:03:53.230713 3913 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:03:53.231047 3915 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:03:53.230659 3920 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:03:53.231933 3906 server_base.cc:1034] running on GCE node
I20260811 19:03:53.232751 3906 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:03:53.234161 3906 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:03:53.235431 3906 hybrid_clock.cc:648] HybridClock initialized: now 1786475033235379 us; error 48 us; skew 500 ppm
I20260811 19:03:53.235788 3906 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:03:53 dist-test-slave-h6n0 krb5kdc[3889](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475033, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:03:53.244148 3906 init.cc:377] Logged in from keytab as kudu/127.29.112.190@KRBTEST.COM (short username kudu)
I20260811 19:03:53.245692 3906 webserver.cc:460] Webserver started at http://127.29.112.190:35187/ using document root <none> and password file <none>
I20260811 19:03:53.246414 3906 fs_manager.cc:359] Metadata directory not provided
I20260811 19:03:53.246568 3906 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:03:53.246932 3906 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:03:53.250005 3906 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474615205622-30146-0/minicluster-data/master-0/data/instance:
uuid: "54aab6d9f02f427f9629f012af265026"
format_stamp: "Formatted at 2026-08-11 19:03:53 on dist-test-slave-h6n0"
server_key: "8ab7167bdaca71d27852205d3b899dcb"
server_key_iv: "9dfd71faaf3fc6d5ebaf2f6a2e9e55ff"
server_key_version: "encryptionkey@0"
I20260811 19:03:53.250924 3906 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474615205622-30146-0/minicluster-data/master-0/wal/instance:
uuid: "54aab6d9f02f427f9629f012af265026"
format_stamp: "Formatted at 2026-08-11 19:03:53 on dist-test-slave-h6n0"
server_key: "8ab7167bdaca71d27852205d3b899dcb"
server_key_iv: "9dfd71faaf3fc6d5ebaf2f6a2e9e55ff"
server_key_version: "encryptionkey@0"
I20260811 19:03:53.256223 3906 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260811 19:03:53.260528 3931 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:03:53.262291 3906 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.005s
I20260811 19:03:53.262504 3906 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474615205622-30146-0/minicluster-data/master-0/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474615205622-30146-0/minicluster-data/master-0/wal
uuid: "54aab6d9f02f427f9629f012af265026"
format_stamp: "Formatted at 2026-08-11 19:03:53 on dist-test-slave-h6n0"
server_key: "8ab7167bdaca71d27852205d3b899dcb"
server_key_iv: "9dfd71faaf3fc6d5ebaf2f6a2e9e55ff"
server_key_version: "encryptionkey@0"
I20260811 19:03:53.262811 3906 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474615205622-30146-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474615205622-30146-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474615205622-30146-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:03:53.280447 3906 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:03:53.285494 3906 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:03:53.285816 3906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:03:53.307508 3906 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.190:33515
I20260811 19:03:53.307607 4007 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.190:33515 every 8 connection(s)
I20260811 19:03:53.309002 3906 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474615205622-30146-0/minicluster-data/master-0/data/info.pb
I20260811 19:03:53.313046 4010 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:03:53.319052 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 3906
I20260811 19:03:53.319278 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474615205622-30146-0/minicluster-data/master-0/wal/instance
I20260811 19:03:53.319725 30146 external_mini_cluster.cc:1428] Setting key a09d3c51f0e05bf852780a7711a3b7e1
I20260811 19:03:53.319984 4010 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.112.190" port: 33515 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 19:03:53 dist-test-slave-h6n0 krb5kdc[3889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475033, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.190@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 19:03:53.346941 4014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:53.323163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:33515 (local address 127.0.0.1:47010)
0811 19:03:53.324099 (+ 936us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:53.324150 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 19:03:53.326956 (+ 2806us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:03:53.327839 (+ 883us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:03:53.327852 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:03:53.328735 (+ 883us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:03:53.329252 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:03:53.329287 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:03:53.331003 (+ 1716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:03:53.331011 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:03:53.332891 (+ 1880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:03:53.332910 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:03:53.333883 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:03:53.333893 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:03:53.334118 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:53.334942 (+ 824us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:03:53.334964 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:03:53.337981 (+ 3017us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:03:53.341322 (+ 3341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:03:53.341336 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:03:53.341347 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:03:53.342186 (+ 839us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:03:53.342737 (+ 551us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:03:53.342743 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:03:53.342747 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:03:53.342932 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:03:53.343783 (+ 851us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:03:53.343792 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:03:53.344025 (+ 233us) client_negotiation.cc:769] Sending connection context
0811 19:03:53.344291 (+ 266us) client_negotiation.cc:240] Negotiation successful
0811 19:03:53.345058 (+ 767us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":641,"mutex_wait_us":195,"thread_start_us":387,"threads_started":1}
I20260811 19:03:53.346972 4015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:53.323163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47010 (local address 127.29.112.190:33515)
0811 19:03:53.325031 (+ 1868us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:53.325038 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:53.325080 (+ 42us) server_negotiation.cc:407] Connection header received
0811 19:03:53.327365 (+ 2285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:53.327371 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:53.327607 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:53.327769 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:53.329516 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:53.330839 (+ 1323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:53.333060 (+ 2221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:53.333627 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:53.333764 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:53.338223 (+ 4459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:03:53.338230 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:03:53.338242 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:03:53.338271 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:03:53.341194 (+ 2923us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:53.342345 (+ 1151us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:53.342356 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:53.342363 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:53.342612 (+ 249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:53.343055 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:53.343061 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:53.343064 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:53.343624 (+ 560us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:03:53.343860 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:53.344437 (+ 577us) server_negotiation.cc:299] Negotiation successful
0811 19:03:53.344905 (+ 468us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":69,"server-negotiator.queue_time_us":1686,"spinlock_wait_cycles":16128,"thread_start_us":1435,"threads_started":1}
I20260811 19:03:53.350234 4010 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.112.189" port: 34961 } has no permanent_uuid. Determining permanent_uuid...
I20260811 19:03:53.351974 4014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:53.351421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.189:34961 (local address 127.0.0.1:57018)
0811 19:03:53.351648 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:53.351812 (+ 164us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.189:34961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
W20260811 19:03:53.352371 3934 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.112.189:34961: connect: Connection refused (error 111)
I20260811 19:03:53.354414 4014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:53.353910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.189:34961 (local address 127.0.0.1:57022)
0811 19:03:53.354070 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:53.354157 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.189:34961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
WARNING: no policy specified for kudu/127.29.112.189@KRBTEST.COM; defaulting to no policy
W20260811 19:03:53.354727 4010 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.112.189:34961: Network error: Client connection negotiation failed: client connection to 127.29.112.189:34961: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.189.keytab.
Entry for principal kudu/127.29.112.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.189.keytab.
I20260811 19:03:53.396031 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474615205622-30146-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474615205622-30146-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474615205622-30146-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474615205622-30146-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.189:34961
--webserver_interface=127.29.112.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.189.keytab
--principal=kudu/127.29.112.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:33881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.190:33515,127.29.112.189:34961,127.29.112.188:44625
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 19:03:53.404196 4010 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.112.189" port: 34961 } attempt: 1
I20260811 19:03:53.405978 4014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:53.405242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.189:34961 (local address 127.0.0.1:57036)
0811 19:03:53.405552 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:53.405761 (+ 209us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.189:34961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260811 19:03:53.407588 4014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:53.406985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.189:34961 (local address 127.0.0.1:57052)
0811 19:03:53.407213 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:53.407344 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.189:34961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260811 19:03:53.407930 4010 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.112.189:34961: Network error: Client connection negotiation failed: client connection to 127.29.112.189:34961: connect: Connection refused (error 111)
I20260811 19:03:53.476229 4010 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.112.189" port: 34961 } attempt: 2
I20260811 19:03:53.477572 4014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:53.477038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.189:34961 (local address 127.0.0.1:57068)
0811 19:03:53.477317 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:53.477421 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.189:34961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260811 19:03:53.478706 4014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:53.478250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.189:34961 (local address 127.0.0.1:57076)
0811 19:03:53.478428 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:53.478500 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.189:34961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 19:03:53.479120 4010 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.112.189:34961: Network error: Client connection negotiation failed: client connection to 127.29.112.189:34961: connect: Connection refused (error 111)
I20260811 19:03:53.570583 4010 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.112.189" port: 34961 } attempt: 3
I20260811 19:03:53.571686 4014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:53.571253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.189:34961 (local address 127.0.0.1:57084)
0811 19:03:53.571466 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:53.571551 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.189:34961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260811 19:03:53.572640 4014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:53.572293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.189:34961 (local address 127.0.0.1:57096)
0811 19:03:53.572445 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:53.572510 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.189:34961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 19:03:53.572815 4010 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.112.189:34961: Network error: Client connection negotiation failed: client connection to 127.29.112.189:34961: connect: Connection refused (error 111)
I20260811 19:03:53.716081 4010 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.112.189" port: 34961 } attempt: 4
I20260811 19:03:53.717295 4014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:53.716748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.189:34961 (local address 127.0.0.1:57102)
0811 19:03:53.717055 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:53.717165 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.189:34961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154}
I20260811 19:03:53.718232 4014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:53.717898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.189:34961 (local address 127.0.0.1:57104)
0811 19:03:53.718044 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:53.718120 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.189:34961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 19:03:53.718407 4010 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.112.189:34961: Network error: Client connection negotiation failed: client connection to 127.29.112.189:34961: connect: Connection refused (error 111)
W20260811 19:03:53.745754 4021 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 19:03:53.746136 4021 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:03:53.746263 4021 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:03:53.746402 4021 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:03:53.754046 4021 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:03:53.754185 4021 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:03:53.754289 4021 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:03:53.754393 4021 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:03:53.754483 4021 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:03:53.764315 4021 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:33881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474615205622-30146-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474615205622-30146-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.29.112.190:33515,127.29.112.189:34961,127.29.112.188:44625
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.189.keytab
--principal=kudu/127.29.112.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.189:34961
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474615205622-30146-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.29.112.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474615205622-30146-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:03:53.765611 4021 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:03:53.767138 4021 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:03:53.779134 4032 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:03:53.779114 4034 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:03:53.779628 4036 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:03:53.779928 4021 server_base.cc:1034] running on GCE node
I20260811 19:03:53.780661 4021 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:03:53.781982 4021 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:03:53.783200 4021 hybrid_clock.cc:648] HybridClock initialized: now 1786475033783122 us; error 51 us; skew 500 ppm
I20260811 19:03:53.783528 4021 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:03:53 dist-test-slave-h6n0 krb5kdc[3889](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475033, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:03:53.786679 4021 init.cc:377] Logged in from keytab as kudu/127.29.112.189@KRBTEST.COM (short username kudu)
I20260811 19:03:53.788198 4021 webserver.cc:460] Webserver started at http://127.29.112.189:44981/ using document root <none> and password file <none>
I20260811 19:03:53.788883 4021 fs_manager.cc:359] Metadata directory not provided
I20260811 19:03:53.789000 4021 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:03:53.789322 4021 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:03:53.793087 4021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474615205622-30146-0/minicluster-data/master-1/data/instance:
uuid: "60f467868d484e4a93fec52bb2bfaadb"
format_stamp: "Formatted at 2026-08-11 19:03:53 on dist-test-slave-h6n0"
server_key: "5f0d19ccb0355be7be946c18cf4d9310"
server_key_iv: "26deb8b6c4dbce6a5ac32774dce166c7"
server_key_version: "encryptionkey@0"
I20260811 19:03:53.793921 4021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474615205622-30146-0/minicluster-data/master-1/wal/instance:
uuid: "60f467868d484e4a93fec52bb2bfaadb"
format_stamp: "Formatted at 2026-08-11 19:03:53 on dist-test-slave-h6n0"
server_key: "5f0d19ccb0355be7be946c18cf4d9310"
server_key_iv: "26deb8b6c4dbce6a5ac32774dce166c7"
server_key_version: "encryptionkey@0"
I20260811 19:03:53.798995 4021 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260811 19:03:53.802419 4044 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:03:53.803772 4021 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 19:03:53.803918 4021 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474615205622-30146-0/minicluster-data/master-1/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474615205622-30146-0/minicluster-data/master-1/wal
uuid: "60f467868d484e4a93fec52bb2bfaadb"
format_stamp: "Formatted at 2026-08-11 19:03:53 on dist-test-slave-h6n0"
server_key: "5f0d19ccb0355be7be946c18cf4d9310"
server_key_iv: "26deb8b6c4dbce6a5ac32774dce166c7"
server_key_version: "encryptionkey@0"
I20260811 19:03:53.804104 4021 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474615205622-30146-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474615205622-30146-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474615205622-30146-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:03:53.821501 4021 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:03:53.825430 4021 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:03:53.825703 4021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:03:53.846607 4021 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.189:34961
I20260811 19:03:53.846628 4136 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.189:34961 every 8 connection(s)
I20260811 19:03:53.848135 4021 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474615205622-30146-0/minicluster-data/master-1/data/info.pb
I20260811 19:03:53.849548 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 4021
I20260811 19:03:53.849735 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474615205622-30146-0/minicluster-data/master-1/wal/instance
I20260811 19:03:53.850318 30146 external_mini_cluster.cc:1428] Setting key 752733e69a1f71cd94be4632e567b93a
I20260811 19:03:53.852887 4138 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:03:53.862746 4138 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.112.190" port: 33515 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 19:03:53 dist-test-slave-h6n0 krb5kdc[3889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475033, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.189@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.29.112.188@KRBTEST.COM; defaulting to no policy
I20260811 19:03:53.884704 4144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:53.866065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47024 (local address 127.29.112.190:33515)
0811 19:03:53.867366 (+ 1301us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:53.867372 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:53.868481 (+ 1109us) server_negotiation.cc:407] Connection header received
0811 19:03:53.870396 (+ 1915us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:53.870402 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:53.870493 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:53.870654 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:53.872176 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:53.872922 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:53.874396 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:53.874865 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:53.875011 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:53.878798 (+ 3787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:03:53.878803 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:03:53.878806 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:03:53.878843 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:03:53.881621 (+ 2778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:53.882347 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:53.882353 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:53.882357 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:53.882480 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:53.882925 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:53.882931 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:53.882934 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:53.883209 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:03:53.883372 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:53.884101 (+ 729us) server_negotiation.cc:299] Negotiation successful
0811 19:03:53.884319 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1190,"thread_start_us":400,"threads_started":1}
I20260811 19:03:53.885636 4145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:53.867028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:33515 (local address 127.0.0.1:47024)
0811 19:03:53.868332 (+ 1304us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:53.868372 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 19:03:53.870134 (+ 1762us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:03:53.870895 (+ 761us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:03:53.870903 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:03:53.871426 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:03:53.871962 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:03:53.871979 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:03:53.873096 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:03:53.873100 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:03:53.874253 (+ 1153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:03:53.874265 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:03:53.875102 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:03:53.875110 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:03:53.875342 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:53.876037 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:03:53.876059 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:03:53.878522 (+ 2463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:03:53.881769 (+ 3247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:03:53.881778 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:03:53.881794 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:03:53.882225 (+ 431us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:03:53.882618 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:03:53.882623 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:03:53.882626 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:03:53.882809 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:03:53.883357 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:03:53.883365 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:03:53.883747 (+ 382us) client_negotiation.cc:769] Sending connection context
0811 19:03:53.884009 (+ 262us) client_negotiation.cc:240] Negotiation successful
0811 19:03:53.884508 (+ 499us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1008,"thread_start_us":604,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.112.188@KRBTEST.COM" created.
I20260811 19:03:53.887404 4138 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.112.189" port: 34961 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 19:03:53 dist-test-slave-h6n0 krb5kdc[3889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475033, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.189@KRBTEST.COM for kudu/127.29.112.189@KRBTEST.COM
I20260811 19:03:53.903869 4146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:53.888807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57112 (local address 127.29.112.189:34961)
0811 19:03:53.889430 (+ 623us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:53.889441 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:53.889475 (+ 34us) server_negotiation.cc:407] Connection header received
0811 19:03:53.889533 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:53.889556 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:53.889787 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:53.889943 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:53.890706 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:53.892353 (+ 1647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:53.893505 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:53.894416 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:53.894593 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:53.897794 (+ 3201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:03:53.897810 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:03:53.897827 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:03:53.897869 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:03:53.900882 (+ 3013us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:53.901715 (+ 833us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:53.901721 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:53.901723 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:53.901833 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:53.902251 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:53.902254 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:53.902256 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:53.902660 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:03:53.902820 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:53.903289 (+ 469us) server_negotiation.cc:299] Negotiation successful
0811 19:03:53.903586 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":93,"server-negotiator.queue_time_us":426,"thread_start_us":270,"threads_started":1}
I20260811 19:03:53.903970 4145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:53.888336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.189:34961 (local address 127.0.0.1:57112)
0811 19:03:53.888789 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:53.888808 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 19:03:53.888909 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:03:53.889975 (+ 1066us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:03:53.889979 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:03:53.890333 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:03:53.890481 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:03:53.890489 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:03:53.892509 (+ 2020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:03:53.892513 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:03:53.893355 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:03:53.893374 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:03:53.894617 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:03:53.894621 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:03:53.894784 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:53.895369 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:03:53.895387 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:03:53.897571 (+ 2184us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:03:53.901021 (+ 3450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:03:53.901029 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:03:53.901036 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:03:53.901589 (+ 553us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:03:53.901940 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:03:53.901955 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:03:53.901962 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:03:53.902147 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:03:53.902794 (+ 647us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:03:53.902803 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:03:53.903000 (+ 197us) client_negotiation.cc:769] Sending connection context
0811 19:03:53.903156 (+ 156us) client_negotiation.cc:240] Negotiation successful
0811 19:03:53.903475 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"mutex_wait_us":28}
I20260811 19:03:53.906502 4138 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.112.188" port: 44625 } has no permanent_uuid. Determining permanent_uuid...
I20260811 19:03:53.908138 4145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:53.907663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.188:44625 (local address 127.0.0.1:40582)
0811 19:03:53.907866 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:53.907972 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.188:44625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260811 19:03:53.908484 4055 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.112.188:44625: connect: Connection refused (error 111)
I20260811 19:03:53.910135 4145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:53.909757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.188:44625 (local address 127.0.0.1:40596)
0811 19:03:53.909943 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:53.910032 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.188:44625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260811 19:03:53.910455 4138 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.112.188:44625: Network error: Client connection negotiation failed: client connection to 127.29.112.188:44625: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.112.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.188.keytab.
Entry for principal kudu/127.29.112.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.188.keytab.
I20260811 19:03:53.927552 30146 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
/tmp/dist-test-taskFzFlwO/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474615205622-30146-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474615205622-30146-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474615205622-30146-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474615205622-30146-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.188:44625
--webserver_interface=127.29.112.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.188.keytab
--principal=kudu/127.29.112.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.112.148:33881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.112.190:33515,127.29.112.189:34961,127.29.112.188:44625
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474615205622-30146-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474615205622-30146-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474615205622-30146-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 19:03:53.959877 4138 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.112.188" port: 44625 } attempt: 1
I20260811 19:03:53.961236 4145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:53.960709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.188:44625 (local address 127.0.0.1:40600)
0811 19:03:53.960951 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:53.961071 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.188:44625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
I20260811 19:03:53.962363 4145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:53.961978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.188:44625 (local address 127.0.0.1:40614)
0811 19:03:53.962148 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:53.962245 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.188:44625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
W20260811 19:03:53.962672 4138 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.112.188:44625: Network error: Client connection negotiation failed: client connection to 127.29.112.188:44625: connect: Connection refused (error 111)
I20260811 19:03:54.017645 4010 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.112.189" port: 34961 } attempt: 5
Aug 11 19:03:54 dist-test-slave-h6n0 krb5kdc[3889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475033, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.190@KRBTEST.COM for kudu/127.29.112.189@KRBTEST.COM
I20260811 19:03:54.030889 4138 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.112.188" port: 44625 } attempt: 2
I20260811 19:03:54.032173 4145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:54.031569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.188:44625 (local address 127.0.0.1:40616)
0811 19:03:54.031870 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:54.031972 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.188:44625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163}
I20260811 19:03:54.033439 4145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:54.032985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.188:44625 (local address 127.0.0.1:40620)
0811 19:03:54.033196 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:54.033284 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.188:44625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119}
W20260811 19:03:54.033772 4138 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.112.188:44625: Network error: Client connection negotiation failed: client connection to 127.29.112.188:44625: connect: Connection refused (error 111)
I20260811 19:03:54.036113 4014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:54.018480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.189:34961 (local address 127.0.0.1:57116)
0811 19:03:54.018822 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:54.018845 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 19:03:54.018992 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:03:54.019652 (+ 660us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:03:54.019661 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:03:54.020288 (+ 627us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:03:54.020603 (+ 315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:03:54.020636 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:03:54.021687 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:03:54.021695 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:03:54.023294 (+ 1599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:03:54.023314 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:03:54.024588 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:03:54.024596 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:03:54.024847 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:54.025854 (+ 1007us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:03:54.025889 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:03:54.028875 (+ 2986us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:03:54.033341 (+ 4466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:03:54.033348 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:03:54.033355 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:03:54.033919 (+ 564us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:03:54.034410 (+ 491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:03:54.034415 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:03:54.034419 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:03:54.034641 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:03:54.035265 (+ 624us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:03:54.035270 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:03:54.035437 (+ 167us) client_negotiation.cc:769] Sending connection context
0811 19:03:54.035604 (+ 167us) client_negotiation.cc:240] Negotiation successful
0811 19:03:54.035874 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221}
I20260811 19:03:54.037077 4146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:54.018620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57116 (local address 127.29.112.189:34961)
0811 19:03:54.018837 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:54.018845 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:54.018908 (+ 63us) server_negotiation.cc:407] Connection header received
0811 19:03:54.019167 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:54.019175 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:54.019273 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:54.019382 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:54.020801 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:54.021537 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:54.023507 (+ 1970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:54.024237 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:54.024367 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:54.029097 (+ 4730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:03:54.029106 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:03:54.029111 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:03:54.029163 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:03:54.033163 (+ 4000us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:54.034087 (+ 924us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:54.034094 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:54.034097 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:54.034256 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:54.034819 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:54.034824 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:54.034828 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:54.035139 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:03:54.035301 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:54.036516 (+ 1215us) server_negotiation.cc:299] Negotiation successful
0811 19:03:54.036729 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 19:03:54.038123 4010 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.112.188" port: 44625 } has no permanent_uuid. Determining permanent_uuid...
I20260811 19:03:54.039254 4014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:54.038888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.188:44625 (local address 127.0.0.1:40628)
0811 19:03:54.039048 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:54.039132 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.188:44625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260811 19:03:54.040197 4014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:54.039858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.188:44625 (local address 127.0.0.1:40638)
0811 19:03:54.040025 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:54.040094 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.188:44625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260811 19:03:54.040441 4010 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.112.188:44625: Network error: Client connection negotiation failed: client connection to 127.29.112.188:44625: connect: Connection refused (error 111)
I20260811 19:03:54.091689 4010 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.112.188" port: 44625 } attempt: 1
I20260811 19:03:54.092845 4014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:54.092375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.188:44625 (local address 127.0.0.1:40652)
0811 19:03:54.092588 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:54.092682 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.188:44625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260811 19:03:54.093773 4014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:54.093466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.188:44625 (local address 127.0.0.1:40660)
0811 19:03:54.093610 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:54.093672 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.188:44625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260811 19:03:54.094008 4010 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.112.188:44625: Network error: Client connection negotiation failed: client connection to 127.29.112.188:44625: connect: Connection refused (error 111)
I20260811 19:03:54.125010 4138 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.112.188" port: 44625 } attempt: 3
I20260811 19:03:54.126157 4145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:54.125689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.188:44625 (local address 127.0.0.1:40672)
0811 19:03:54.125890 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:54.125993 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.188:44625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260811 19:03:54.127411 4145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:54.126936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.188:44625 (local address 127.0.0.1:40680)
0811 19:03:54.127201 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:54.127276 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.188:44625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151}
W20260811 19:03:54.127619 4138 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.112.188:44625: Network error: Client connection negotiation failed: client connection to 127.29.112.188:44625: connect: Connection refused (error 111)
I20260811 19:03:54.162259 4010 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.112.188" port: 44625 } attempt: 2
I20260811 19:03:54.163564 4014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:54.163044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.188:44625 (local address 127.0.0.1:40682)
0811 19:03:54.163293 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:54.163379 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.188:44625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260811 19:03:54.164501 4014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:54.164191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.188:44625 (local address 127.0.0.1:40690)
0811 19:03:54.164325 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:54.164397 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.188:44625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 19:03:54.164702 4010 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.112.188:44625: Network error: Client connection negotiation failed: client connection to 127.29.112.188:44625: connect: Connection refused (error 111)
I20260811 19:03:54.270869 4010 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.112.188" port: 44625 } attempt: 3
I20260811 19:03:54.270900 4138 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.112.188" port: 44625 } attempt: 4
I20260811 19:03:54.272344 4145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:54.271699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.188:44625 (local address 127.0.0.1:40712)
0811 19:03:54.271978 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:54.272123 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.188:44625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260811 19:03:54.273036 4014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:54.271657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.188:44625 (local address 127.0.0.1:40700)
0811 19:03:54.272590 (+ 933us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:54.272749 (+ 159us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.188:44625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260811 19:03:54.273761 4145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:54.273360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.188:44625 (local address 127.0.0.1:40714)
0811 19:03:54.273512 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:54.273605 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.188:44625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260811 19:03:54.274304 4138 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.112.188:44625: Network error: Client connection negotiation failed: client connection to 127.29.112.188:44625: connect: Connection refused (error 111)
I20260811 19:03:54.274909 4014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:54.274341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.188:44625 (local address 127.0.0.1:40722)
0811 19:03:54.274648 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:54.274756 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.112.188:44625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144}
W20260811 19:03:54.275194 4010 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.112.188:44625: Network error: Client connection negotiation failed: client connection to 127.29.112.188:44625: connect: Connection refused (error 111)
W20260811 19:03:54.312114 4150 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 19:03:54.312487 4150 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:03:54.312626 4150 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:03:54.312742 4150 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:03:54.320271 4150 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:03:54.320369 4150 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:03:54.320461 4150 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:03:54.320545 4150 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:03:54.320653 4150 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:03:54.330933 4150 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.112.148:33881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474615205622-30146-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474615205622-30146-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.29.112.190:33515,127.29.112.189:34961,127.29.112.188:44625
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474615205622-30146-0/krb5kdc/kudu_127.29.112.188.keytab
--principal=kudu/127.29.112.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.112.188:44625
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474615205622-30146-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.29.112.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474615205622-30146-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:03:54.332687 4150 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:03:54.334033 4150 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:03:54.346442 4157 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:03:54.346554 4161 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:03:54.346675 4158 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:03:54.347993 4150 server_base.cc:1034] running on GCE node
I20260811 19:03:54.349229 4150 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:03:54.351204 4150 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:03:54.352530 4150 hybrid_clock.cc:648] HybridClock initialized: now 1786475034352452 us; error 41 us; skew 500 ppm
I20260811 19:03:54.352968 4150 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:03:54 dist-test-slave-h6n0 krb5kdc[3889](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475034, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:03:54.356823 4150 init.cc:377] Logged in from keytab as kudu/127.29.112.188@KRBTEST.COM (short username kudu)
I20260811 19:03:54.358968 4150 webserver.cc:460] Webserver started at http://127.29.112.188:35365/ using document root <none> and password file <none>
I20260811 19:03:54.359745 4150 fs_manager.cc:359] Metadata directory not provided
I20260811 19:03:54.360016 4150 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:03:54.360525 4150 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:03:54.364452 4150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474615205622-30146-0/minicluster-data/master-2/data/instance:
uuid: "d17124c95c884c6dbb29ff8e11783b4c"
format_stamp: "Formatted at 2026-08-11 19:03:54 on dist-test-slave-h6n0"
server_key: "c2ecc725670ad690284b783f0d564a55"
server_key_iv: "f61875d2fe5329822cf325779c533712"
server_key_version: "encryptionkey@0"
I20260811 19:03:54.365568 4150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474615205622-30146-0/minicluster-data/master-2/wal/instance:
uuid: "d17124c95c884c6dbb29ff8e11783b4c"
format_stamp: "Formatted at 2026-08-11 19:03:54 on dist-test-slave-h6n0"
server_key: "c2ecc725670ad690284b783f0d564a55"
server_key_iv: "f61875d2fe5329822cf325779c533712"
server_key_version: "encryptionkey@0"
I20260811 19:03:54.371160 4150 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 19:03:54.375078 4174 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:03:54.376531 4150 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 19:03:54.376734 4150 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474615205622-30146-0/minicluster-data/master-2/data,/tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474615205622-30146-0/minicluster-data/master-2/wal
uuid: "d17124c95c884c6dbb29ff8e11783b4c"
format_stamp: "Formatted at 2026-08-11 19:03:54 on dist-test-slave-h6n0"
server_key: "c2ecc725670ad690284b783f0d564a55"
server_key_iv: "f61875d2fe5329822cf325779c533712"
server_key_version: "encryptionkey@0"
I20260811 19:03:54.376972 4150 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474615205622-30146-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474615205622-30146-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474615205622-30146-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:03:54.412124 4150 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:03:54.417768 4150 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:03:54.418089 4150 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:03:54.439311 4150 rpc_server.cc:307] RPC server started. Bound to: 127.29.112.188:44625
I20260811 19:03:54.439356 4260 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.112.188:44625 every 8 connection(s)
I20260811 19:03:54.440773 4150 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474615205622-30146-0/minicluster-data/master-2/data/info.pb
I20260811 19:03:54.444342 30146 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu as pid 4150
I20260811 19:03:54.444521 4261 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:03:54.444556 30146 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFzFlwO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474615205622-30146-0/minicluster-data/master-2/wal/instance
I20260811 19:03:54.445008 30146 external_mini_cluster.cc:1428] Setting key e8c6ed0f4d20fcba02615215277c607f
I20260811 19:03:54.452471 4010 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.112.188" port: 44625 } attempt: 4
I20260811 19:03:54.456059 4261 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.112.190" port: 33515 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 19:03:54 dist-test-slave-h6n0 krb5kdc[3889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475032, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.188@KRBTEST.COM
Aug 11 19:03:54 dist-test-slave-h6n0 krb5kdc[3889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475033, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.190@KRBTEST.COM for kudu/127.29.112.188@KRBTEST.COM
Aug 11 19:03:54 dist-test-slave-h6n0 krb5kdc[3889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475034, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.188@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 19:03:54.472087 4014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:54.453184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.188:44625 (local address 127.0.0.1:40736)
0811 19:03:54.453380 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:54.453404 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 19:03:54.453502 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:03:54.454420 (+ 918us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:03:54.454425 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:03:54.455001 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:03:54.455192 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:03:54.455204 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:03:54.457457 (+ 2253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:03:54.457463 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:03:54.458934 (+ 1471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:03:54.458951 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:03:54.460123 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:03:54.460129 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:03:54.460326 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:54.461781 (+ 1455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:03:54.461799 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:03:54.464570 (+ 2771us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:03:54.469283 (+ 4713us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:03:54.469289 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:03:54.469293 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:03:54.469709 (+ 416us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:03:54.470117 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:03:54.470122 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:03:54.470125 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:03:54.470257 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:03:54.470936 (+ 679us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:03:54.470941 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:03:54.471124 (+ 183us) client_negotiation.cc:769] Sending connection context
0811 19:03:54.471294 (+ 170us) client_negotiation.cc:240] Negotiation successful
0811 19:03:54.471592 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260811 19:03:54.472779 4266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:54.446814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40724 (local address 127.29.112.188:44625)
0811 19:03:54.447742 (+ 928us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:54.447758 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:54.447807 (+ 49us) server_negotiation.cc:407] Connection header received
0811 19:03:54.449569 (+ 1762us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:54.449596 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:54.450098 (+ 502us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:54.450523 (+ 425us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:54.452000 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:54.454013 (+ 2013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:54.455764 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:54.456616 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:54.456820 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:54.462718 (+ 5898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:03:54.462725 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:03:54.462734 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:03:54.462767 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:03:54.466867 (+ 4100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:54.468241 (+ 1374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:54.468250 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:54.468260 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:54.468505 (+ 245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:54.468945 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:54.468948 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:54.468950 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:54.469603 (+ 653us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:03:54.469874 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:54.470902 (+ 1028us) server_negotiation.cc:299] Negotiation successful
0811 19:03:54.471251 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":39,"server-negotiator.queue_time_us":671,"thread_start_us":397,"threads_started":1}
I20260811 19:03:54.472779 4267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:54.453317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40736 (local address 127.29.112.188:44625)
0811 19:03:54.454084 (+ 767us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:54.454090 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:54.454117 (+ 27us) server_negotiation.cc:407] Connection header received
0811 19:03:54.454168 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:54.454173 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:54.454243 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:54.454367 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:54.455555 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:54.457222 (+ 1667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:54.459080 (+ 1858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:54.459917 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:54.460080 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:54.464803 (+ 4723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:03:54.464808 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:03:54.464811 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:03:54.464839 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:03:54.469161 (+ 4322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:54.469848 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:54.469853 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:54.469857 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:54.469996 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:54.470377 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:54.470382 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:54.470385 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:54.470813 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:03:54.470955 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:54.471419 (+ 464us) server_negotiation.cc:299] Negotiation successful
0811 19:03:54.471605 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":65,"server-negotiator.queue_time_us":599,"thread_start_us":394,"threads_started":1}
I20260811 19:03:54.480939 4269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:54.459100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.190:33515 (local address 127.0.0.1:47028)
0811 19:03:54.461357 (+ 2257us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:54.461400 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 19:03:54.461559 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:03:54.461984 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:03:54.461992 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:03:54.462843 (+ 851us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:03:54.463264 (+ 421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:03:54.463281 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:03:54.465611 (+ 2330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:03:54.465616 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:03:54.467638 (+ 2022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:03:54.467657 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:03:54.468619 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:03:54.468628 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:03:54.468845 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:54.469963 (+ 1118us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:03:54.469994 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:03:54.472902 (+ 2908us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:03:54.477986 (+ 5084us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:03:54.477998 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:03:54.478005 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:03:54.478425 (+ 420us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:03:54.478843 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:03:54.478848 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:03:54.478852 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:03:54.479092 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:03:54.479753 (+ 661us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:03:54.479761 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:03:54.479955 (+ 194us) client_negotiation.cc:769] Sending connection context
0811 19:03:54.480156 (+ 201us) client_negotiation.cc:240] Negotiation successful
0811 19:03:54.480505 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2004,"thread_start_us":1722,"threads_started":1}
I20260811 19:03:54.481108 4268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:54.459005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47028 (local address 127.29.112.190:33515)
0811 19:03:54.459569 (+ 564us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:54.459574 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:54.461526 (+ 1952us) server_negotiation.cc:407] Connection header received
0811 19:03:54.461691 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:54.461696 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:54.461777 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:54.461900 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:54.463465 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:54.465397 (+ 1932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:54.467826 (+ 2429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:54.468399 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:54.468538 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:54.473414 (+ 4876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:03:54.473419 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:03:54.473422 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:03:54.473451 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:03:54.477829 (+ 4378us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:54.478596 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:54.478601 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:54.478604 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:54.478729 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:54.479237 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:54.479242 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:54.479250 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:54.479591 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:03:54.479768 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:54.480283 (+ 515us) server_negotiation.cc:299] Negotiation successful
0811 19:03:54.480515 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":409,"thread_start_us":266,"threads_started":1}
I20260811 19:03:54.481288 4010 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54aab6d9f02f427f9629f012af265026: Bootstrap starting.
I20260811 19:03:54.483368 4261 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.112.189" port: 34961 } has no permanent_uuid. Determining permanent_uuid...
I20260811 19:03:54.486883 4010 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 54aab6d9f02f427f9629f012af265026: Neither blocks nor log segments found. Creating new log.
Aug 11 19:03:54 dist-test-slave-h6n0 krb5kdc[3889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475032, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.190@KRBTEST.COM
I20260811 19:03:54.488464 4010 log.cc:824] T 00000000000000000000000000000000 P 54aab6d9f02f427f9629f012af265026: Log is configured to *not* fsync() on all Append() calls
Aug 11 19:03:54 dist-test-slave-h6n0 krb5kdc[3889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475034, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.188@KRBTEST.COM for kudu/127.29.112.189@KRBTEST.COM
I20260811 19:03:54.493963 4010 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54aab6d9f02f427f9629f012af265026: No bootstrap required, opened a new log
I20260811 19:03:54.498625 4269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:54.484095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.189:34961 (local address 127.0.0.1:57120)
0811 19:03:54.484340 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:54.484357 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 19:03:54.484459 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:03:54.484797 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:03:54.484801 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:03:54.485187 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:03:54.485319 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:03:54.485328 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:03:54.486292 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:03:54.486297 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:03:54.487409 (+ 1112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:03:54.487426 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:03:54.488151 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:03:54.488157 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:03:54.488330 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:54.488903 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:03:54.488920 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:03:54.491614 (+ 2694us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:03:54.495675 (+ 4061us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:03:54.495682 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:03:54.495686 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:03:54.496197 (+ 511us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:03:54.496743 (+ 546us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:03:54.496748 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:03:54.496751 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:03:54.496940 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:03:54.497603 (+ 663us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:03:54.497609 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:03:54.497773 (+ 164us) client_negotiation.cc:769] Sending connection context
0811 19:03:54.497933 (+ 160us) client_negotiation.cc:240] Negotiation successful
0811 19:03:54.498223 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135}
I20260811 19:03:54.498945 4146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:54.484179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57120 (local address 127.29.112.189:34961)
0811 19:03:54.484374 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:54.484382 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:54.484404 (+ 22us) server_negotiation.cc:407] Connection header received
0811 19:03:54.484586 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:54.484591 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:54.484670 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:54.484780 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:54.485463 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:54.486145 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:54.487556 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:54.487982 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:54.488134 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:54.491958 (+ 3824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:03:54.491965 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:03:54.491969 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:03:54.492024 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:03:54.495519 (+ 3495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:54.496375 (+ 856us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:54.496383 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:54.496387 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:54.496591 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:54.497068 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:54.497073 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:54.497077 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:54.497447 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:03:54.497601 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:54.498051 (+ 450us) server_negotiation.cc:299] Negotiation successful
0811 19:03:54.498285 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 19:03:54.499440 4272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:54.475944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47044 (local address 127.29.112.190:33515)
0811 19:03:54.476616 (+ 672us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:54.476620 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:54.476640 (+ 20us) server_negotiation.cc:407] Connection header received
0811 19:03:54.476685 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:54.476688 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:54.476741 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:54.476873 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:54.478164 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:54.479461 (+ 1297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:54.481289 (+ 1828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:54.482063 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:54.482196 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:54.489051 (+ 6855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:03:54.489150 (+ 99us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:03:54.489159 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:03:54.489211 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:03:54.493674 (+ 4463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:54.495531 (+ 1857us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:54.495538 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:54.495541 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:54.495970 (+ 429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:54.496492 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:54.496498 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:54.496506 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:54.496954 (+ 448us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:03:54.497219 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:54.498442 (+ 1223us) server_negotiation.cc:299] Negotiation successful
0811 19:03:54.498951 (+ 509us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":573,"thread_start_us":343,"threads_started":1}
I20260811 19:03:54.499948 4261 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.112.188" port: 44625 } has no permanent_uuid. Determining permanent_uuid...
I20260811 19:03:54.504758 4010 raft_consensus.cc:348] T 00000000000000000000000000000000 P 54aab6d9f02f427f9629f012af265026 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54aab6d9f02f427f9629f012af265026" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 33515 } } peers { permanent_uuid: "60f467868d484e4a93fec52bb2bfaadb" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 34961 } } peers { permanent_uuid: "d17124c95c884c6dbb29ff8e11783b4c" member_type: VOTER last_known_addr { host: "127.29.112.188" port: 44625 } }
I20260811 19:03:54.505270 4010 raft_consensus.cc:729] T 00000000000000000000000000000000 P 54aab6d9f02f427f9629f012af265026 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54aab6d9f02f427f9629f012af265026, State: Initialized, Role: FOLLOWER
I20260811 19:03:54.505805 4010 consensus_queue.cc:260] T 00000000000000000000000000000000 P 54aab6d9f02f427f9629f012af265026 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54aab6d9f02f427f9629f012af265026" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 33515 } } peers { permanent_uuid: "60f467868d484e4a93fec52bb2bfaadb" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 34961 } } peers { permanent_uuid: "d17124c95c884c6dbb29ff8e11783b4c" member_type: VOTER last_known_addr { host: "127.29.112.188" port: 44625 } }
Aug 11 19:03:54 dist-test-slave-h6n0 krb5kdc[3889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475034, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.188@KRBTEST.COM for kudu/127.29.112.188@KRBTEST.COM
I20260811 19:03:54.508157 4275 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54aab6d9f02f427f9629f012af265026 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54aab6d9f02f427f9629f012af265026" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 33515 } } peers { permanent_uuid: "60f467868d484e4a93fec52bb2bfaadb" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 34961 } } peers { permanent_uuid: "d17124c95c884c6dbb29ff8e11783b4c" member_type: VOTER last_known_addr { host: "127.29.112.188" port: 44625 } } }
I20260811 19:03:54.508606 4275 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54aab6d9f02f427f9629f012af265026 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 19:03:54.509595 4010 sys_catalog.cc:564] T 00000000000000000000000000000000 P 54aab6d9f02f427f9629f012af265026 [sys.catalog]: configured and running, proceeding with master startup.
Aug 11 19:03:54 dist-test-slave-h6n0 krb5kdc[3889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475032, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.112.189@KRBTEST.COM
I20260811 19:03:54.517247 4269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:54.500801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.188:44625 (local address 127.0.0.1:40738)
0811 19:03:54.500962 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:54.500980 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 19:03:54.501090 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:03:54.501419 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:03:54.501423 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:03:54.501893 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:03:54.502105 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:03:54.502123 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:03:54.503551 (+ 1428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:03:54.503558 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:03:54.504908 (+ 1350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:03:54.504921 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:03:54.505728 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:03:54.505733 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:03:54.505961 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:54.506572 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:03:54.506595 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:03:54.509079 (+ 2484us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:03:54.513492 (+ 4413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:03:54.513499 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:03:54.513504 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:03:54.514374 (+ 870us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:03:54.514952 (+ 578us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:03:54.514960 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:03:54.514965 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:03:54.515115 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:03:54.516165 (+ 1050us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:03:54.516172 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:03:54.516369 (+ 197us) client_negotiation.cc:769] Sending connection context
0811 19:03:54.516570 (+ 201us) client_negotiation.cc:240] Negotiation successful
0811 19:03:54.516935 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51,"mutex_wait_us":5}
I20260811 19:03:54.518343 4266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:54.501037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40738 (local address 127.29.112.188:44625)
0811 19:03:54.501192 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:54.501197 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:54.501212 (+ 15us) server_negotiation.cc:407] Connection header received
0811 19:03:54.501254 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:54.501257 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:54.501305 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:54.501400 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:54.502272 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:54.503392 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:54.505057 (+ 1665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:54.505586 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:54.505720 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:54.509253 (+ 3533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:03:54.509259 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:03:54.509263 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:03:54.509296 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:03:54.513336 (+ 4040us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:54.514615 (+ 1279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:54.514621 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:54.514625 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:54.514826 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:54.515325 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:54.515333 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:54.515340 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:54.515933 (+ 593us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:03:54.516090 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:54.517769 (+ 1679us) server_negotiation.cc:299] Negotiation successful
0811 19:03:54.518019 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":34,"server-negotiator.queue_time_us":54}
I20260811 19:03:54.517656 4295 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 19:03:54.523056 4146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:54.502624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57126 (local address 127.29.112.189:34961)
0811 19:03:54.502786 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:54.502790 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:54.502803 (+ 13us) server_negotiation.cc:407] Connection header received
0811 19:03:54.502840 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:54.502842 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:54.502892 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:54.502995 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:54.504540 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:54.505660 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:54.507931 (+ 2271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:54.508534 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:54.508637 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:54.515753 (+ 7116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:03:54.515762 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:03:54.515769 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:03:54.515823 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:03:54.519499 (+ 3676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:54.520207 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:54.520213 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:54.520216 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:54.520362 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:54.521010 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:54.521015 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:54.521019 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:54.521363 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:03:54.521649 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:54.522237 (+ 588us) server_negotiation.cc:299] Negotiation successful
0811 19:03:54.522682 (+ 445us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 19:03:54.523878 4261 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d17124c95c884c6dbb29ff8e11783b4c: Bootstrap starting.
I20260811 19:03:54.528527 4261 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d17124c95c884c6dbb29ff8e11783b4c: Neither blocks nor log segments found. Creating new log.
I20260811 19:03:54.529161 30146 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 19:03:54.530124 4261 log.cc:824] T 00000000000000000000000000000000 P d17124c95c884c6dbb29ff8e11783b4c: Log is configured to *not* fsync() on all Append() calls
I20260811 19:03:54.532866 3973 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "54aab6d9f02f427f9629f012af265026"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47044
I20260811 19:03:54.533278 3973 raft_consensus.cc:482] T 00000000000000000000000000000000 P 54aab6d9f02f427f9629f012af265026 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260811 19:03:54.533403 4261 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d17124c95c884c6dbb29ff8e11783b4c: No bootstrap required, opened a new log
I20260811 19:03:54.533489 3973 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 54aab6d9f02f427f9629f012af265026 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:03:54.537159 3973 raft_consensus.cc:504] T 00000000000000000000000000000000 P 54aab6d9f02f427f9629f012af265026 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54aab6d9f02f427f9629f012af265026" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 33515 } } peers { permanent_uuid: "60f467868d484e4a93fec52bb2bfaadb" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 34961 } } peers { permanent_uuid: "d17124c95c884c6dbb29ff8e11783b4c" member_type: VOTER last_known_addr { host: "127.29.112.188" port: 44625 } }
W20260811 19:03:54.538784 3942 tablet.cc:2369] T 00000000000000000000000000000000 P 54aab6d9f02f427f9629f012af265026: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 19:03:54.539719 3973 leader_election.cc:288] T 00000000000000000000000000000000 P 54aab6d9f02f427f9629f012af265026 [CANDIDATE]: Term 1 election: Requested vote from peers 60f467868d484e4a93fec52bb2bfaadb (127.29.112.189:34961), d17124c95c884c6dbb29ff8e11783b4c (127.29.112.188:44625)
I20260811 19:03:54.528868 3906 master_runner.cc:186] Error getting master registration for 127.29.112.189:34961: OK, instance_id { permanent_uuid: "60f467868d484e4a93fec52bb2bfaadb" instance_seqno: 1786475033840147 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 19:03:54.540467 3906 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 19:03:54.541872 4261 raft_consensus.cc:348] T 00000000000000000000000000000000 P d17124c95c884c6dbb29ff8e11783b4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54aab6d9f02f427f9629f012af265026" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 33515 } } peers { permanent_uuid: "60f467868d484e4a93fec52bb2bfaadb" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 34961 } } peers { permanent_uuid: "d17124c95c884c6dbb29ff8e11783b4c" member_type: VOTER last_known_addr { host: "127.29.112.188" port: 44625 } }
I20260811 19:03:54.542325 4261 raft_consensus.cc:729] T 00000000000000000000000000000000 P d17124c95c884c6dbb29ff8e11783b4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d17124c95c884c6dbb29ff8e11783b4c, State: Initialized, Role: FOLLOWER
I20260811 19:03:54.542850 4261 consensus_queue.cc:260] T 00000000000000000000000000000000 P d17124c95c884c6dbb29ff8e11783b4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54aab6d9f02f427f9629f012af265026" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 33515 } } peers { permanent_uuid: "60f467868d484e4a93fec52bb2bfaadb" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 34961 } } peers { permanent_uuid: "d17124c95c884c6dbb29ff8e11783b4c" member_type: VOTER last_known_addr { host: "127.29.112.188" port: 44625 } }
I20260811 19:03:54.544113 4302 sys_catalog.cc:455] T 00000000000000000000000000000000 P d17124c95c884c6dbb29ff8e11783b4c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54aab6d9f02f427f9629f012af265026" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 33515 } } peers { permanent_uuid: "60f467868d484e4a93fec52bb2bfaadb" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 34961 } } peers { permanent_uuid: "d17124c95c884c6dbb29ff8e11783b4c" member_type: VOTER last_known_addr { host: "127.29.112.188" port: 44625 } } }
I20260811 19:03:54.544970 4302 sys_catalog.cc:458] T 00000000000000000000000000000000 P d17124c95c884c6dbb29ff8e11783b4c [sys.catalog]: This master's current role is: FOLLOWER
I20260811 19:03:54.545300 4261 sys_catalog.cc:564] T 00000000000000000000000000000000 P d17124c95c884c6dbb29ff8e11783b4c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:03:54.545374 4220 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "54aab6d9f02f427f9629f012af265026" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "d17124c95c884c6dbb29ff8e11783b4c"
I20260811 19:03:54.546000 4220 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d17124c95c884c6dbb29ff8e11783b4c [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:03:54.550706 4220 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d17124c95c884c6dbb29ff8e11783b4c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54aab6d9f02f427f9629f012af265026 in term 1.
I20260811 19:03:54.551476 3939 leader_election.cc:302] T 00000000000000000000000000000000 P 54aab6d9f02f427f9629f012af265026 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54aab6d9f02f427f9629f012af265026, d17124c95c884c6dbb29ff8e11783b4c; no voters:
I20260811 19:03:54.552037 4275 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 54aab6d9f02f427f9629f012af265026 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:03:54.573540 4138 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.112.188" port: 44625 } attempt: 5
I20260811 19:03:54.586068 4272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:54.553288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47050 (local address 127.29.112.190:33515)
0811 19:03:54.553537 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:54.553546 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:54.565493 (+ 11947us) server_negotiation.cc:407] Connection header received
0811 19:03:54.566067 (+ 574us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:54.566076 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:54.566188 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:54.566280 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:54.570288 (+ 4008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:54.571499 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:54.573642 (+ 2143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:54.574372 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:54.574484 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:54.578556 (+ 4072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:03:54.578566 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:03:54.578571 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:03:54.578614 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:03:54.582403 (+ 3789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:54.583351 (+ 948us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:54.583357 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:54.583360 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:54.583527 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:54.584054 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:54.584061 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:54.584064 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:54.584451 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:03:54.584680 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:54.585340 (+ 660us) server_negotiation.cc:299] Negotiation successful
0811 19:03:54.585599 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
Aug 11 19:03:54 dist-test-slave-h6n0 krb5kdc[3889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475033, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.112.189@KRBTEST.COM for kudu/127.29.112.188@KRBTEST.COM
I20260811 19:03:54.598207 4145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:54.575007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.112.188:44625 (local address 127.0.0.1:40760)
0811 19:03:54.575521 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0811 19:03:54.575550 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0811 19:03:54.575705 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:03:54.576395 (+ 690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:03:54.576402 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:03:54.577101 (+ 699us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:03:54.577359 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:03:54.577375 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:03:54.579136 (+ 1761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:03:54.579145 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:03:54.581321 (+ 2176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:03:54.581345 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:03:54.583183 (+ 1838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:03:54.583194 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:03:54.583673 (+ 479us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:54.584605 (+ 932us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:03:54.584654 (+ 49us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:03:54.588731 (+ 4077us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:03:54.594168 (+ 5437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:03:54.594177 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:03:54.594184 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:03:54.594833 (+ 649us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:03:54.595405 (+ 572us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:03:54.595414 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:03:54.595420 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:03:54.595672 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:03:54.596876 (+ 1204us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:03:54.596884 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:03:54.597087 (+ 203us) client_negotiation.cc:769] Sending connection context
0811 19:03:54.597281 (+ 194us) client_negotiation.cc:240] Negotiation successful
0811 19:03:54.597653 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337}
I20260811 19:03:54.599988 4267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:54.575132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40760 (local address 127.29.112.188:44625)
0811 19:03:54.575389 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:54.575396 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:54.575854 (+ 458us) server_negotiation.cc:407] Connection header received
0811 19:03:54.575936 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:54.575944 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:54.576042 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:54.576150 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:54.577573 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:54.578930 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:54.581607 (+ 2677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:54.582731 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:54.582907 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:54.588999 (+ 6092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:03:54.589009 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:03:54.589017 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:03:54.589069 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:03:54.593961 (+ 4892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:54.595020 (+ 1059us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:54.595029 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:54.595035 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:54.595244 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:54.595828 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:54.595836 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:54.595842 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:54.596297 (+ 455us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:03:54.596469 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:54.599158 (+ 2689us) server_negotiation.cc:299] Negotiation successful
0811 19:03:54.599478 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72,"spinlock_wait_cycles":58752}
I20260811 19:03:54.599197 4146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:54.553579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57136 (local address 127.29.112.189:34961)
0811 19:03:54.553827 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:54.553835 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:54.570041 (+ 16206us) server_negotiation.cc:407] Connection header received
0811 19:03:54.570273 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:54.570279 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:54.570383 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:54.570483 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:54.574118 (+ 3635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:54.576166 (+ 2048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:54.579811 (+ 3645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:54.584360 (+ 4549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:54.584460 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:54.589890 (+ 5430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:03:54.589898 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:03:54.589904 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:03:54.589948 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:03:54.593915 (+ 3967us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:54.595184 (+ 1269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:54.595190 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:54.595193 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:54.595352 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:54.595840 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:54.595845 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:54.595848 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:54.596503 (+ 655us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:03:54.596863 (+ 360us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:54.597909 (+ 1046us) server_negotiation.cc:299] Negotiation successful
0811 19:03:54.598660 (+ 751us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 19:03:54.604990 4327 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 19:03:54.606114 4266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:54.552819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40754 (local address 127.29.112.188:44625)
0811 19:03:54.553001 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:54.553008 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:54.588319 (+ 35311us) server_negotiation.cc:407] Connection header received
0811 19:03:54.588613 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:54.588619 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:54.588707 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:54.588784 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:54.591433 (+ 2649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:54.592666 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:54.594567 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:54.595296 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:54.595420 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:54.598960 (+ 3540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:03:54.598967 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:03:54.598972 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:03:54.599013 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:03:54.602657 (+ 3644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:54.603512 (+ 855us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:54.603518 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:54.603521 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:54.603691 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:03:54.604163 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:03:54.604169 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:03:54.604171 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:03:54.604559 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:03:54.604897 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:54.605520 (+ 623us) server_negotiation.cc:299] Negotiation successful
0811 19:03:54.605756 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 19:03:54.607223 4275 raft_consensus.cc:686] T 00000000000000000000000000000000 P 54aab6d9f02f427f9629f012af265026 [term 1 LEADER]: Becoming Leader. State: Replica: 54aab6d9f02f427f9629f012af265026, State: Running, Role: LEADER
I20260811 19:03:54.607808 4275 consensus_queue.cc:237] T 00000000000000000000000000000000 P 54aab6d9f02f427f9629f012af265026 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54aab6d9f02f427f9629f012af265026" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 33515 } } peers { permanent_uuid: "60f467868d484e4a93fec52bb2bfaadb" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 34961 } } peers { permanent_uuid: "d17124c95c884c6dbb29ff8e11783b4c" member_type: VOTER last_known_addr { host: "127.29.112.188" port: 44625 } }
I20260811 19:03:54.610607 4138 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60f467868d484e4a93fec52bb2bfaadb: Bootstrap starting.
I20260811 19:03:54.612568 4307 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54aab6d9f02f427f9629f012af265026 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 54aab6d9f02f427f9629f012af265026. Latest consensus state: current_term: 1 leader_uuid: "54aab6d9f02f427f9629f012af265026" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54aab6d9f02f427f9629f012af265026" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 33515 } } peers { permanent_uuid: "60f467868d484e4a93fec52bb2bfaadb" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 34961 } } peers { permanent_uuid: "d17124c95c884c6dbb29ff8e11783b4c" member_type: VOTER last_known_addr { host: "127.29.112.188" port: 44625 } } }
I20260811 19:03:54.613373 4307 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54aab6d9f02f427f9629f012af265026 [sys.catalog]: This master's current role is: LEADER
I20260811 19:03:54.614720 4138 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 60f467868d484e4a93fec52bb2bfaadb: Neither blocks nor log segments found. Creating new log.
I20260811 19:03:54.614718 4332 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:03:54.616098 4138 log.cc:824] T 00000000000000000000000000000000 P 60f467868d484e4a93fec52bb2bfaadb: Log is configured to *not* fsync() on all Append() calls
I20260811 19:03:54.621515 4138 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60f467868d484e4a93fec52bb2bfaadb: No bootstrap required, opened a new log
I20260811 19:03:54.616796 4150 master_runner.cc:186] Error getting master registration for 127.29.112.189:34961: OK, instance_id { permanent_uuid: "60f467868d484e4a93fec52bb2bfaadb" instance_seqno: 1786475033840147 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 19:03:54.626842 4150 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 19:03:54.634009 4332 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:03:54.635057 4138 raft_consensus.cc:348] T 00000000000000000000000000000000 P 60f467868d484e4a93fec52bb2bfaadb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54aab6d9f02f427f9629f012af265026" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 33515 } } peers { permanent_uuid: "60f467868d484e4a93fec52bb2bfaadb" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 34961 } } peers { permanent_uuid: "d17124c95c884c6dbb29ff8e11783b4c" member_type: VOTER last_known_addr { host: "127.29.112.188" port: 44625 } }
I20260811 19:03:54.635458 4138 raft_consensus.cc:729] T 00000000000000000000000000000000 P 60f467868d484e4a93fec52bb2bfaadb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60f467868d484e4a93fec52bb2bfaadb, State: Initialized, Role: FOLLOWER
I20260811 19:03:54.635980 4138 consensus_queue.cc:260] T 00000000000000000000000000000000 P 60f467868d484e4a93fec52bb2bfaadb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54aab6d9f02f427f9629f012af265026" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 33515 } } peers { permanent_uuid: "60f467868d484e4a93fec52bb2bfaadb" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 34961 } } peers { permanent_uuid: "d17124c95c884c6dbb29ff8e11783b4c" member_type: VOTER last_known_addr { host: "127.29.112.188" port: 44625 } }
I20260811 19:03:54.645591 4220 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d17124c95c884c6dbb29ff8e11783b4c [term 1 FOLLOWER]: Refusing update from remote peer 54aab6d9f02f427f9629f012af265026: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:03:54.646780 4275 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 54aab6d9f02f427f9629f012af265026 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d17124c95c884c6dbb29ff8e11783b4c" member_type: VOTER last_known_addr { host: "127.29.112.188" port: 44625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:03:54.647867 4138 sys_catalog.cc:564] T 00000000000000000000000000000000 P 60f467868d484e4a93fec52bb2bfaadb [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:03:54.648600 4089 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 60f467868d484e4a93fec52bb2bfaadb [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:03:54.652737 4089 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 60f467868d484e4a93fec52bb2bfaadb [term 1 FOLLOWER]: Refusing update from remote peer 54aab6d9f02f427f9629f012af265026: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:03:54.653226 4091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "54aab6d9f02f427f9629f012af265026" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "60f467868d484e4a93fec52bb2bfaadb"
I20260811 19:03:54.654033 4275 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 54aab6d9f02f427f9629f012af265026 [LEADER]: Connected to new peer: Peer: permanent_uuid: "60f467868d484e4a93fec52bb2bfaadb" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 34961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:03:54.656903 4091 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 60f467868d484e4a93fec52bb2bfaadb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54aab6d9f02f427f9629f012af265026 in term 1.
I20260811 19:03:54.663888 4337 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60f467868d484e4a93fec52bb2bfaadb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54aab6d9f02f427f9629f012af265026" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 33515 } } peers { permanent_uuid: "60f467868d484e4a93fec52bb2bfaadb" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 34961 } } peers { permanent_uuid: "d17124c95c884c6dbb29ff8e11783b4c" member_type: VOTER last_known_addr { host: "127.29.112.188" port: 44625 } } }
I20260811 19:03:54.666131 4337 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60f467868d484e4a93fec52bb2bfaadb [sys.catalog]: This master's current role is: FOLLOWER
W20260811 19:03:54.670445 4187 tablet.cc:2369] T 00000000000000000000000000000000 P d17124c95c884c6dbb29ff8e11783b4c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 19:03:54.674669 4302 sys_catalog.cc:455] T 00000000000000000000000000000000 P d17124c95c884c6dbb29ff8e11783b4c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 54aab6d9f02f427f9629f012af265026. Latest consensus state: current_term: 1 leader_uuid: "54aab6d9f02f427f9629f012af265026" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54aab6d9f02f427f9629f012af265026" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 33515 } } peers { permanent_uuid: "60f467868d484e4a93fec52bb2bfaadb" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 34961 } } peers { permanent_uuid: "d17124c95c884c6dbb29ff8e11783b4c" member_type: VOTER last_known_addr { host: "127.29.112.188" port: 44625 } } }
I20260811 19:03:54.675099 4302 sys_catalog.cc:458] T 00000000000000000000000000000000 P d17124c95c884c6dbb29ff8e11783b4c [sys.catalog]: This master's current role is: FOLLOWER
I20260811 19:03:54.676518 4338 mvcc.cc:205] Tried to move back new op lower bound from 7317401741897699328 to 7317401741758877696. Current Snapshot: MvccSnapshot[applied={T|T < 7317401741897699328}]
I20260811 19:03:54.679380 4340 mvcc.cc:205] Tried to move back new op lower bound from 7317401741897699328 to 7317401741758877696. Current Snapshot: MvccSnapshot[applied={T|T < 7317401741897699328}]
I20260811 19:03:54.686791 4302 sys_catalog.cc:455] T 00000000000000000000000000000000 P d17124c95c884c6dbb29ff8e11783b4c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "54aab6d9f02f427f9629f012af265026" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54aab6d9f02f427f9629f012af265026" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 33515 } } peers { permanent_uuid: "60f467868d484e4a93fec52bb2bfaadb" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 34961 } } peers { permanent_uuid: "d17124c95c884c6dbb29ff8e11783b4c" member_type: VOTER last_known_addr { host: "127.29.112.188" port: 44625 } } }
I20260811 19:03:54.687063 4302 sys_catalog.cc:458] T 00000000000000000000000000000000 P d17124c95c884c6dbb29ff8e11783b4c [sys.catalog]: This master's current role is: FOLLOWER
I20260811 19:03:54.691968 4337 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60f467868d484e4a93fec52bb2bfaadb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 54aab6d9f02f427f9629f012af265026. Latest consensus state: current_term: 1 leader_uuid: "54aab6d9f02f427f9629f012af265026" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54aab6d9f02f427f9629f012af265026" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 33515 } } peers { permanent_uuid: "60f467868d484e4a93fec52bb2bfaadb" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 34961 } } peers { permanent_uuid: "d17124c95c884c6dbb29ff8e11783b4c" member_type: VOTER last_known_addr { host: "127.29.112.188" port: 44625 } } }
I20260811 19:03:54.692312 4337 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60f467868d484e4a93fec52bb2bfaadb [sys.catalog]: This master's current role is: FOLLOWER
I20260811 19:03:54.693261 4307 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54aab6d9f02f427f9629f012af265026 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "54aab6d9f02f427f9629f012af265026" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54aab6d9f02f427f9629f012af265026" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 33515 } } peers { permanent_uuid: "60f467868d484e4a93fec52bb2bfaadb" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 34961 } } peers { permanent_uuid: "d17124c95c884c6dbb29ff8e11783b4c" member_type: VOTER last_known_addr { host: "127.29.112.188" port: 44625 } } }
I20260811 19:03:54.693547 4307 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54aab6d9f02f427f9629f012af265026 [sys.catalog]: This master's current role is: LEADER
I20260811 19:03:54.694315 4332 catalog_manager.cc:1347] Generated new cluster ID: 8e750c9546234a14a923d40f3d20e658
I20260811 19:03:54.694430 4332 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:03:54.694672 4307 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54aab6d9f02f427f9629f012af265026 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "54aab6d9f02f427f9629f012af265026" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54aab6d9f02f427f9629f012af265026" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 33515 } } peers { permanent_uuid: "60f467868d484e4a93fec52bb2bfaadb" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 34961 } } peers { permanent_uuid: "d17124c95c884c6dbb29ff8e11783b4c" member_type: VOTER last_known_addr { host: "127.29.112.188" port: 44625 } } }
I20260811 19:03:54.695055 4307 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54aab6d9f02f427f9629f012af265026 [sys.catalog]: This master's current role is: LEADER
I20260811 19:03:54.704038 4348 mvcc.cc:205] Tried to move back new op lower bound from 7317401741897699328 to 7317401741758877696. Current Snapshot: MvccSnapshot[applied={T|T < 7317401741897699328}]
I20260811 19:03:54.706002 4337 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60f467868d484e4a93fec52bb2bfaadb [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "54aab6d9f02f427f9629f012af265026" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54aab6d9f02f427f9629f012af265026" member_type: VOTER last_known_addr { host: "127.29.112.190" port: 33515 } } peers { permanent_uuid: "60f467868d484e4a93fec52bb2bfaadb" member_type: VOTER last_known_addr { host: "127.29.112.189" port: 34961 } } peers { permanent_uuid: "d17124c95c884c6dbb29ff8e11783b4c" member_type: VOTER last_known_addr { host: "127.29.112.188" port: 44625 } } }
I20260811 19:03:54.706321 4337 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60f467868d484e4a93fec52bb2bfaadb [sys.catalog]: This master's current role is: FOLLOWER
I20260811 19:03:54.722800 4365 catalog_manager.cc:1259] Loaded cluster ID: 8e750c9546234a14a923d40f3d20e658
I20260811 19:03:54.723030 4365 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 60f467868d484e4a93fec52bb2bfaadb: loading cluster ID for follower catalog manager: success
W20260811 19:03:54.724385 4365 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 60f467868d484e4a93fec52bb2bfaadb: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20260811 19:03:54.724606 4365 catalog_manager.cc:873] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
I20260811 19:03:54.737475 4332 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:03:54.739643 4332 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:03:54.764283 4332 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 54aab6d9f02f427f9629f012af265026: Generated new TSK 0
I20260811 19:03:54.765528 4332 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 19:03:54.897653 4146 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:03:54.882667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57144 (local address 127.29.112.189:34961)
0811 19:03:54.882944 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:54.882954 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:54.895267 (+ 12313us) server_negotiation.cc:407] Connection header received
0811 19:03:54.895614 (+ 347us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:54.895621 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:54.895732 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:54.895817 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:54.897375 (+ 1558us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57144: BlockingRecv error: recv got EOF from 127.0.0.1:57144 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260811 19:03:54.897804 4272 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:03:54.882485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47066 (local address 127.29.112.190:33515)
0811 19:03:54.882901 (+ 416us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:54.882908 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:54.895640 (+ 12732us) server_negotiation.cc:407] Connection header received
0811 19:03:54.895777 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:54.895783 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:54.895894 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:54.896021 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:54.897528 (+ 1507us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47066: BlockingRecv error: recv got EOF from 127.0.0.1:47066 (error 108)
Metrics: {"server-negotiator.queue_time_us":265}
W20260811 19:03:54.898260 4266 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:03:54.889847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40770 (local address 127.29.112.188:44625)
0811 19:03:54.890040 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:54.890048 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:54.896248 (+ 6200us) server_negotiation.cc:407] Connection header received
0811 19:03:54.896568 (+ 320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:54.896576 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:54.896676 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:54.896752 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:54.898005 (+ 1253us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40770: BlockingRecv error: recv got EOF from 127.0.0.1:40770 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260811 19:03:54.905793 4146 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:03:54.903386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57160 (local address 127.29.112.189:34961)
0811 19:03:54.903566 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:54.903571 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:54.904029 (+ 458us) server_negotiation.cc:407] Connection header received
0811 19:03:54.904343 (+ 314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:54.904348 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:54.904475 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:54.904559 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:54.905494 (+ 935us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57160: BlockingRecv error: recv got EOF from 127.0.0.1:57160 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
W20260811 19:03:54.914314 4146 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:03:54.912002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57162 (local address 127.29.112.189:34961)
0811 19:03:54.912167 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:54.912173 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:54.912641 (+ 468us) server_negotiation.cc:407] Connection header received
0811 19:03:54.912930 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:54.912936 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:54.913022 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:54.913105 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:54.914086 (+ 981us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57162: BlockingRecv error: recv got EOF from 127.0.0.1:57162 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260811 19:03:54.924871 4146 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:03:54.920799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57178 (local address 127.29.112.189:34961)
0811 19:03:54.920996 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:54.921003 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:54.923097 (+ 2094us) server_negotiation.cc:407] Connection header received
0811 19:03:54.923391 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:54.923399 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:54.923515 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:54.923600 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:54.924589 (+ 989us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57178: BlockingRecv error: recv got EOF from 127.0.0.1:57178 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
W20260811 19:03:54.935869 4146 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:03:54.933166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57182 (local address 127.29.112.189:34961)
0811 19:03:54.933340 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:54.933347 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:54.933838 (+ 491us) server_negotiation.cc:407] Connection header received
0811 19:03:54.934329 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:54.934335 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:54.934432 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:54.934513 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:54.935615 (+ 1102us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57182: BlockingRecv error: recv got EOF from 127.0.0.1:57182 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260811 19:03:54.951206 4146 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:03:54.948935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57192 (local address 127.29.112.189:34961)
0811 19:03:54.949136 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:54.949143 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:54.949692 (+ 549us) server_negotiation.cc:407] Connection header received
0811 19:03:54.949907 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:54.949914 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:54.950011 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:54.950136 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:54.950952 (+ 816us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57192: BlockingRecv error: recv got EOF from 127.0.0.1:57192 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
W20260811 19:03:54.974160 4146 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:03:54.971614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57194 (local address 127.29.112.189:34961)
0811 19:03:54.971839 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:54.971846 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:54.972433 (+ 587us) server_negotiation.cc:407] Connection header received
0811 19:03:54.972627 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:54.972633 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:54.972736 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:54.972853 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:54.973897 (+ 1044us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57194: BlockingRecv error: recv got EOF from 127.0.0.1:57194 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
W20260811 19:03:55.016181 4146 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:03:55.013429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57196 (local address 127.29.112.189:34961)
0811 19:03:55.013639 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:55.013648 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:55.014294 (+ 646us) server_negotiation.cc:407] Connection header received
0811 19:03:55.014692 (+ 398us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:55.014698 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:55.014823 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:55.014945 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:55.015876 (+ 931us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57196: BlockingRecv error: recv got EOF from 127.0.0.1:57196 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
W20260811 19:03:55.095027 4146 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:03:55.092440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57200 (local address 127.29.112.189:34961)
0811 19:03:55.092686 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:55.092694 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:55.093281 (+ 587us) server_negotiation.cc:407] Connection header received
0811 19:03:55.093543 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:55.093549 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:55.093648 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:55.093771 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:55.094776 (+ 1005us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57200: BlockingRecv error: recv got EOF from 127.0.0.1:57200 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
W20260811 19:03:55.228971 4146 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:03:55.226855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57204 (local address 127.29.112.189:34961)
0811 19:03:55.227120 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:55.227127 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:55.227441 (+ 314us) server_negotiation.cc:407] Connection header received
0811 19:03:55.227666 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:55.227670 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:55.227743 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:55.227896 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:55.228663 (+ 767us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57204: BlockingRecv error: recv got EOF from 127.0.0.1:57204 (error 108)
Metrics: {"server-negotiator.queue_time_us":94}
W20260811 19:03:55.492271 4146 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:03:55.489843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57208 (local address 127.29.112.189:34961)
0811 19:03:55.490059 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:55.490065 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:55.490375 (+ 310us) server_negotiation.cc:407] Connection header received
0811 19:03:55.490493 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:55.490497 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:55.490652 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:55.490758 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:03:55.491912 (+ 1154us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57208: BlockingRecv error: recv got EOF from 127.0.0.1:57208 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 19:03:55.609863 4327 catalog_manager.cc:1259] Loaded cluster ID: 8e750c9546234a14a923d40f3d20e658
I20260811 19:03:55.609973 4327 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d17124c95c884c6dbb29ff8e11783b4c: loading cluster ID for follower catalog manager: success
I20260811 19:03:55.612679 4327 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d17124c95c884c6dbb29ff8e11783b4c: acquiring CA information for follower catalog manager: success
I20260811 19:03:55.614589 4327 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d17124c95c884c6dbb29ff8e11783b4c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 19:03:55.729740 4365 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 60f467868d484e4a93fec52bb2bfaadb: acquiring CA information for follower catalog manager: success
I20260811 19:03:55.732533 4365 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 60f467868d484e4a93fec52bb2bfaadb: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 19:03:56.019418 4494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:56.009591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57214 (local address 127.29.112.189:34961)
0811 19:03:56.010676 (+ 1085us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:56.010685 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:56.010973 (+ 288us) server_negotiation.cc:407] Connection header received
0811 19:03:56.011298 (+ 325us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:56.011304 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:56.011545 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:56.011690 (+ 145us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:03:56.012433 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:56.013259 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:56.015652 (+ 2393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:56.016427 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:56.016586 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:56.017468 (+ 882us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:03:56.018289 (+ 821us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:03:56.018424 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:56.018932 (+ 508us) server_negotiation.cc:299] Negotiation successful
0811 19:03:56.019025 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":680,"thread_start_us":401,"threads_started":1}
I20260811 19:03:56.034775 4506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:56.028372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40784 (local address 127.29.112.188:44625)
0811 19:03:56.028944 (+ 572us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:56.028950 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:56.028970 (+ 20us) server_negotiation.cc:407] Connection header received
0811 19:03:56.029161 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:56.029165 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:56.029320 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:56.029491 (+ 171us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:03:56.029967 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:56.030807 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:56.032119 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:56.032660 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:56.032827 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:56.033417 (+ 590us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:03:56.033984 (+ 567us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:03:56.034166 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:56.034249 (+ 83us) server_negotiation.cc:299] Negotiation successful
0811 19:03:56.034338 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":429,"thread_start_us":269,"threads_started":1}
I20260811 19:03:56.055117 4494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:56.041169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57222 (local address 127.29.112.189:34961)
0811 19:03:56.041378 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:56.041385 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:56.049080 (+ 7695us) server_negotiation.cc:407] Connection header received
0811 19:03:56.049269 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:56.049277 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:56.049388 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:56.049494 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:03:56.050069 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:56.051173 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:56.053131 (+ 1958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:56.053515 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:56.053650 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:56.054214 (+ 564us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:03:56.054325 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:03:56.054437 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:56.054672 (+ 235us) server_negotiation.cc:299] Negotiation successful
0811 19:03:56.054761 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 19:03:56.057005 4516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:56.040800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47082 (local address 127.29.112.190:33515)
0811 19:03:56.049635 (+ 8835us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:56.049644 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:56.049669 (+ 25us) server_negotiation.cc:407] Connection header received
0811 19:03:56.049821 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:56.049843 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:56.050116 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:56.050329 (+ 213us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:03:56.050997 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:56.051975 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:56.054183 (+ 2208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:56.054871 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:56.054990 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:56.055717 (+ 727us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:03:56.056190 (+ 473us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:03:56.056322 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:56.056565 (+ 243us) server_negotiation.cc:299] Negotiation successful
0811 19:03:56.056667 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8671,"thread_start_us":8549,"threads_started":1}
I20260811 19:03:56.057711 4506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:03:56.041019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40800 (local address 127.29.112.188:44625)
0811 19:03:56.041191 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 19:03:56.041198 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:03:56.051168 (+ 9970us) server_negotiation.cc:407] Connection header received
0811 19:03:56.051438 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:03:56.051443 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:03:56.051526 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:03:56.051619 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:03:56.052453 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:56.053585 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:56.055689 (+ 2104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:03:56.056088 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:03:56.056225 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:03:56.056711 (+ 486us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:03:56.056834 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:03:56.056895 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 19:03:56.057300 (+ 405us) server_negotiation.cc:299] Negotiation successful
0811 19:03:56.057387 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 19:03:56.064844 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 3906
I20260811 19:03:56.067050 4003 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:03:56.192356 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 3906
W20260811 19:03:56.223865 4055 connection.cc:535] server connection from 127.0.0.1:57116 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 19:03:56.224577 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 4021
I20260811 19:03:56.225903 4131 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:03:56.351603 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 4021
I20260811 19:03:56.381628 30146 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu pid 4150
I20260811 19:03:56.383353 4252 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:03:56.496430 30146 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFzFlwO/build/asan/bin/kudu with pid 4150
2026-08-11T19:03:56Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (5426 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (10623 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (421251 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST